Fleet Management Platform for Multi-Purpose Robots

ABSTRACT

A robot fleet management platform includes a robot inventory that indicates robots that can be assigned to a robot fleet and, for each robot, a set of baseline features and a status. A components inventory indicates different components that can be provisioned to one or more multi-purpose robots and, for each component, a set of extended capabilities and a status. The platform receives a request for a robotic fleet to perform a job and determines a job definition data structure defining a set of tasks. The platform determines a respective configuration for each assigned multi-purpose robots based on the respective set of tasks that is assigned to the one or more assigned multi-purpose robots and the components inventory. The platform configures the more assigned multi-purpose robots based on the respective configuration. The platform deploys the robotic fleet including the one or more assigned multi-purpose robots to perform the job.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of PCT App. No.PCT/US22/28633 filed 10 May 2022, which claims priority to India PatentApp. No. 202211008709 filed 18 Feb. 2022 and claims the benefit of Prov.App. No. 63/302,013 filed 21 Jan. 2022, Prov. App. No. 63/299,710 filed14 Jan. 2022, Prov. App. No. 63/282,507 filed 23 Nov. 2021, and Prov.App. No. 63/187,325 filed 11 May 2021.

This application is a continuation-in-part of PCT App. No.PCT/US21/64233 filed 17 Dec. 2021, which claims priority to India PatentApp. No. 202111036187 filed 10 Aug. 2021 and India Patent App. No.202111029964 filed 3 Jul. 2021 and claims the benefit of Prov. App. No.63/185,348 filed 6 May 2021 and Prov. App. No. 63/127,983 filed 18 Dec.2020.

The entire disclosures of the above applications are incorporated byreference.

FIELD

The present disclosure relates to information technology methods andsystems for management of value chain network entities, including supplychain and demand management entities. The present disclosure alsorelates to the field of enterprise management platforms, moreparticularly involving an edge-distributed database and query languagefor storing and retrieving value chain data.

BACKGROUND

Historically, many of the various categories of goods purchased and usedby household consumers, by businesses and by other customers were beensupplied mainly through a relatively linear fashion, in whichmanufacturers and other suppliers of finished goods, components, andother items handed off items to shipping companies, freight forwardersand the like, who delivered them to warehouses for temporary storage, toretailers, where customers purchased them, or directly to customerlocations. Manufacturers and retailers undertook various sales andmarketing activities to encourage and meet demand by customers,including designing products, positioning them on shelves and inadvertising, setting prices, and the like.

Orders for products were fulfilled by manufacturers through a supplychain, such as depicted in FIG. 1 , where suppliers 122 in varioussupply environments 160, operating production facilities 134 or actingas resellers or distributors for others, made a product 130 available ata point of origin 102 in response to an order. The product 130 waspassed through the supply chain, being conveyed and stored via varioushauling facilities 138 and distribution facilities 134, such aswarehouses 132, fulfillment centers 112 and delivery systems 114, suchas trucks and other vehicles, trains, and the like. In many cases,maritime facilities and infrastructure, such as ships, barges, docks andports provided transport over waterways between the points of origin 102and one or more destinations 104.

Organizations have access to an almost unlimited amount of data. Withthe advent of smart connected devices, wearable technologies, theInternet of Things (IoT), and the like, the amount of data available toan organization that is planning, overseeing, managing and operating avalue chain network has increased dramatically and will likely continueto do so. For example, in a manufacturing facility, warehouse, campus,or other operating environment, there may be hundreds to thousands ofIoT sensors that provide metrics such as vibration data that measure thevibration signatures of important machinery, temperatures throughout thefacility, motion sensors that can track throughput, asset trackingsensors and beacons to locate items, cameras and optical sensors,chemical and biological sensors, and many others. Additionally, aswearable technologies become more prevalent, wearables may provideinsight into the movement, health indicators, physiological states,activity states, movements, and other characteristics of workers.Furthermore, as organizations implement CRM systems, ERP systems,operations systems, information technology systems, advanced analyticsand other systems that leverage information and information technology,organizations have access to an increasingly wide array of other largedata sets, such as marketing data, sales data, operational data,information technology data, performance data, customer data, financialdata, market data, pricing data, supply chain data, and the like,including data sets generated by or for the organization and third-partydata sets.

The presence of more data and data of new types offers manyopportunities for organizations to achieve competitive advantages;however, it also presents problems, such as of complexity and volume,such that users can be overwhelmed, missing opportunities for insight. Aneed exists for methods and systems that allow enterprises not only toobtain data, but to convert the data into insights and to translate theinsights into well-informed decisions and timely execution of efficientoperations.

Acquiring large data sets from thousands, or potentially millions ofdevices (containing large numbers of sensors) distributed acrossmultiple organizations in a value chain network has become more typical.For example, there is a proliferation of Radio Frequency Identification(RFID) Tags to individual goods in retail stores. In this situation andother similar situations, a vast number of data streams can overwhelmthe ability to transmit the data across networks and/or the ability tocreate effective automated centralized decisions.

The proliferation of data generators (e.g., sensors) has created anopportunity to manage networks such as value chain networks with inputfrom massive numbers of distributed points of semi-intelligent control.However, current approaches often rely on limited centralized datacollection due to bandwidth, storage, processing, and/or otherlimitations.

SUMMARY

According to some embodiments of the present disclosure, a method forprocessing a query for data stored in a distributed database isdisclosed. The method includes receiving, at an edge device, the queryfor data stored in the distributed database from a query device. Themethod further includes causing, by the edge device, the query to bestored on a dynamic ledger maintained by the distributed database. Themethod further includes detecting, by the edge device, that summary datahas been stored on the dynamic ledger. The method further includesgenerating, by the edge device, an approximate response to the querybased on the summary data stored on the dynamic ledger. The methodfurther includes transmitting, to the query device, the approximateresponse.

In some embodiments, the query is an EDQL query. In some embodiments,the query specifies a shard algorithm, wherein the shard algorithmspecifies a location of data stored in the distributed database. In someembodiments, the dynamic ledger is a blockchain.

In some embodiments, causing the query to be stored on the dynamicledger comprises transmitting, by the edge device, the query to anaggregator. In some of these embodiments, the aggregator is a blockchainnode.

In some embodiments, generating the approximate response to the querybased on the summary data stored on the dynamic ledger further comprisesgenerating, using the summary data, a probability distribution model fordata corresponding to the query; and generating, using the probabilitydistribution model, the approximate response. In some of theseembodiments, the method further includes receiving a second query fordata stored in the distributed database; and generating an approximateresponse to the second query using the probability distribution modelwithout causing the second query to be stored on the dynamic ledger.Additionally or alternatively, the probability distribution model is aneural network, wherein generating the probability distribution modelcomprises training the neural network.

In some embodiments, the method further includes generating a query planbased on the received query. In some of these embodiments, the queryplan comprises transmitting the query to other edge devices, the methodfurther comprising transmitting the query to the other edge devices.Additionally or alternatively, the query plan comprises transmitting thequery to an aggregator, the method further comprising transmitting thequery to the aggregator.

In some embodiments, the method further includes executing the queryagainst edge storage connected to the edge device to obtain partialquery results. In some of these embodiments, the approximate response tothe query is further based on the partial query results.

In some embodiments, the edge device is an edge device/aggregator. Insome embodiments, detecting that summary data has been stored on thedynamic ledger comprises detecting that a threshold percentage of edgedevices have caused summary data to be stored on the dynamic ledger.

In some embodiments, the summary data is generated based on data storedat other edge devices. In some embodiments, the summary data comprisesstatistical data. In some embodiments, the summary data includes outlierdata. In some embodiments, the data is sensor data.

According to some embodiments of the present disclosure, a method forprocessing a query for data stored in a distributed database isdisclosed. The method includes receiving, at an edge device, the queryfor data stored in the distributed database from a query device, whereinthe query is a request for data stored at the edge device and for datastored at other edge devices. The method further includes executing, bythe edge device, the query to find partial query results comprising thedata stored at the edge device. The method further includes generating,by the edge device, statistical information based on the partial queryresults. The method further includes determining, by the edge device, astatistical confidence associated with the partial results based on thestatistical information. The method further includes generating, by theedge device, an approximate response to the query based on thestatistical information. The method further includes transmitting theapproximate response to the query device.

In some embodiments, the query is an EDQL query. In some embodiments,the query specifies a shard algorithm, wherein the shard algorithmspecifies a location of data stored in the distributed database. In someembodiments, the method further includes causing the statisticalinformation to be stored on a dynamic ledger.

In some embodiments, generating the approximate response to the querybased on the statistical information further comprises: generating,using the statistical information, a probability distribution model fordata corresponding to the query; and generating, using the probabilitydistribution model, the approximate response. In some of theseembodiments, the method further includes receiving a second query fordata stored in the distributed database; and generating an approximateresponse to the second query using the probability distribution model.Additionally or alternatively, the probability distribution model is aneural network, wherein generating the probability distribution modelcomprises training the neural network.

In some embodiments, the method further includes generating a query planbased on the received query. In some embodiments, the approximateresponse to the query is further based on the partial query results. Insome embodiments, the edge device is an edge device/aggregator. In someembodiments, the statistical information includes outlier data.

In some embodiments, the data stored at the edge device comprises sensordata. In some of these embodiments, the sensor data is collected fromsensors connected to the edge device. Additionally or alternatively, thesensor data is collected from sensors connected to a different edgedevice.

In some embodiments, the distributed database comprises a mesh networkof edge devices. In some embodiments, the method further includesreceiving an instruction, from an aggregator, to reproduce a subset ofthe data stored at the edge device to a second edge device; andtransmitting the subset of the data to the second edge device.

In some embodiments, the query is a distributed join query. In some ofthese embodiments, generating the partial query results comprises usinga reference table stored at the edge device. In some of theseembodiments, the reference table is a distributed reference table.Additionally or alternatively, the distributed join query is executedwithout network overhead.

According to some embodiments of the present disclosure, a method foroptimizing a distributed database is disclosed. The method includesreceiving, at an aggregator, one or more query logs comprising pastqueries received by the distributed database. The method furtherincludes generating, by the aggregator, a query prediction model basedon the one or more query logs. The method further includes predicting,by the aggregator, a future query using the query prediction model,wherein the future query is predicted to be received by an edge device.The method further includes causing, by the aggregator, data forresponding to the predicted future query to be transmitted to the edgedevice.

In some embodiments, the data for responding to the predicted futurequery comprises data stored at another edge device. In some of theseembodiments, the method further includes locating the data forresponding to the predicted future query suing a sharding algorithm. Insome of these embodiments, the sharding algorithm is a neural networkalgorithm. Additionally or alternatively, the sharding algorithm is agenetic algorithm. Additionally or alternatively, the sharding algorithmis a logical algorithm.

In some embodiments, the data for responding to the predicted futurequery is summary data. In some of these embodiments, the summary datacomprises statistical data. Additionally or alternatively, the summarydata includes outlier data. Additionally or alternatively, the methodfurther includes instructing, by the aggregator, another edge device togenerate the summary data. Additionally or alternatively, the methodfurther includes storing the summary data on a dynamic ledger maintainedby the aggregator. In some of these embodiments, the dynamic ledger is ablockchain.

In some embodiments, data for responding to the predicted future queryis a probability distribution model. In some of these embodiments, themethod further includes generating the probability distribution modelbased on data stored at another edge device. In some of theseembodiments, the method further includes storing the probabilitydistribution model on a dynamic ledger maintained by the aggregator.

In some embodiments, the future query is an EDQL query. In someembodiments, the data for responding to the future query comprisessensor data. In some embodiments, the distributed database comprises amesh network of edge devices.

In some embodiments, the predicted future query is a distributed joinquery. In some of these embodiments, the data for responding to thepredicted future query is a reference table.

According to some embodiments of the present disclosure, a method forprocessing a query for data stored in a distributed database isdisclosed. The method includes monitoring, by an edge device, one ormore pending data requests stored on a dynamic ledger. The methodfurther includes detecting, by the edge device, a pending data requestcomprising a query for data stored in the distributed database, whereinthe query is a request for data stored at the edge device and for datastored at other edge devices. The method further includes executing, bythe edge device, the query to find partial query results comprising thedata stored at the edge device. The method further includes generating,by the edge device, summary data based on the partial query results. Themethod further includes causing, by the edge device, the summary data tobe stored on the dynamic ledger.

In some embodiments, the summary data comprises statistical data. Insome embodiments, the summary data includes outlier data. In someembodiments, the dynamic ledger is a blockchain.

In some embodiments, causing the summary data to be stored on thedynamic ledger comprises transmitting the summary data to an aggregatorresponsible for maintaining the dynamic ledger. In some of theseembodiments, the aggregator is a blockchain node.

In some embodiments, the method further includes generating, based onthe summary data, a probability distribution model; and causing theprobability distribution model to be stored on the dynamic ledger.

In some embodiments, the query is an EDQL query. In some embodiments,the data stored in the distributed database comprises sensor data. Insome embodiments, the distributed database comprises a mesh network ofedge devices.

According to some embodiments of the present disclosure, a method forprocessing a query for data stored in a distributed database isdisclosed. The method includes receiving, at an edge device, the queryfor data stored in the distributed database from a query device, whereinthe query comprises a distributed join referencing at least two tables,wherein the at least two tables are distributed across a plurality ofedge devices comprising the edge device. The method further includesobtaining, by the edge device, one or more distributed reference tables.The method further includes executing, by the edge device, using the oneor more distributed reference tables, the query to find partial queryresults comprising data stored at the edge device. The method furtherincludes generating, by the edge device, an approximate response to thequery using the partial query results.

In some embodiments, the query is an EDQL query. In some embodiments,the query specifies a shard algorithm, wherein the shard algorithmspecifies a location of data stored in the distributed database.

In some embodiments, the distributed reference tables are stored on adynamic ledger. In some of these embodiments, the dynamic ledger is ablockchain. Additionally or alternatively, the method further includescausing the query to be stored on the dynamic ledger by transmitting thequery to an aggregator.

In some embodiments, generating the approximate response to the queryusing the partial query results further comprises: generating, using thepartial query results a probability distribution model for datacorresponding to the query; and generating, using the probabilitydistribution model, the approximate response. In some of theseembodiments, the probability distribution model is a neural network,wherein generating the probability distribution model comprises trainingthe neural network.

In some embodiments, the method further includes generating a query planbased on the received query. In some of these embodiments, the queryplan comprises transmitting the query to other edge devices, the methodfurther comprising transmitting the query to the other edge devices.Additionally or alternatively, the query plan comprises transmitting thequery to an aggregator, the method further comprising transmitting thequery to the aggregator.

In some embodiments, the edge device is an edge device/aggregator.

In some embodiments, the method further includes generating summary databased on the partial query results. In some of these embodiments, thesummary data comprises statistical data. Additionally or alternatively,the summary data includes outlier data. Additionally or alternatively,the data is sensor data.

In some embodiments, the distributed database comprises a mesh networkof edge devices. In some embodiments, the distributed database comprisesa fully connected network of edge devices. In some embodiments, themethod further includes receiving an instruction, from an aggregator, toreproduce a subset of the data stored at the edge device to a secondedge device; and transmitting the subset of the data to the second edgedevice. In some embodiments, the distributed join query is executedwithout network overhead.

According to some embodiments of the present disclosure, a method foroptimizing a distributed database is disclosed. The method includesreceiving, at an aggregator, one or more query logs comprising pastqueries received by the distributed database. The method furtherincludes determining, by the aggregator, common queries received by oneor more edge devices. The method further includes determining, by theaggregator, that at least one edge device was not able to respond to acommon query received by the at least one edge device. The methodfurther includes causing, by the aggregator, data for responding to thecommon query to be transmitted to the at least one edge device.

In some embodiments, the data for responding to the common querycomprises data stored at another edge device. In some of theseembodiments, the method further includes locating the data forresponding to the common query using a sharding algorithm. In some ofthese embodiments, the sharding algorithm is a neural network algorithm.Additionally or alternatively, the sharding algorithm is a geneticalgorithm. Additionally or alternatively, the sharding algorithm is alogical algorithm.

In some embodiments, the data for responding to the predicted futurequery is summary data. In some of these embodiments, the summary datacomprises statistical data. Additionally or alternatively, the summarydata includes outlier data. Additionally or alternatively, the methodfurther includes instructing, by the aggregator, another edge device togenerate the summary data. Additionally or alternatively, the methodfurther includes storing the summary data on a dynamic ledger maintainedby the aggregator. In some of these embodiments, the dynamic ledger is ablockchain.

In some embodiments, the data for responding to the common query is aprobability distribution model. In some of these embodiments, the methodfurther includes generating the probability distribution model based ondata stored at another edge device. In some of these embodiments, themethod further includes storing the probability distribution model on adynamic ledger maintained by the aggregator.

In some embodiments, the common query is an EDQL query. In someembodiments, the data for responding to the common query comprisessensor data. In some embodiments, the distributed database comprises amesh network of edge devices. In some embodiments, the common query is adistributed join query. In some of these embodiments, the data forresponding to the common query is a reference table.

According to some embodiments of the present disclosure, a method forprioritizing predictive model data streams is disclosed. The methodincludes receiving, by a first device, a plurality of predictive modeldata streams, wherein each predictive model data streams comprises a setof model parameters for a corresponding predictive model, wherein eachpredictive model is trained to predict future data values of a datasource. The method further includes prioritizing, by the first device,priorities to each of the plurality of predictive model data streams.The method further includes selecting at least one of the predictivemodel data streams based on a corresponding priority. The method furtherincludes parameterizing, by the first device, a predictive model usingthe set of model parameters included in the selected predictive modelstream. The method further includes predicting, by the first device,future data values of the data source using the parameterized predictivemodel.

In some embodiments, the selected at least one predictive model datastream is associated with a high priority. In some embodiments, theselecting comprises suppressing the predictive model data streams thatwere not selected based on the priorities associated with eachnon-selected predictive model data stream. In some embodiments,assigning priorities to each of the plurality of predictive model datastreams comprises determining whether each set of model parameters isunusual. In some embodiments, assigning priorities to each of theplurality of predictive model data streams comprises determining whethereach set of model parameters has changed from a previous value.

In some embodiments, the set of model parameters comprise at least onevector.

In some of these embodiments, the at least one vector comprises a motionvector associated with a robot. In some of these embodiments, the futuredata values comprise one or more future predicted locations of therobot.

In some embodiments, the predictive model predicts stock levels ofitems, the method further comprising: detecting, based on the futuredata values, an upcoming supply shortage of an item; and taking actionto avoid running out of the item. In some embodiments, the predictivemodel is a behavior analysis model, wherein the future data valuesindicate a predicted behavior of an entity. In some embodiments, thepredictive model is an augmentation model, wherein the future datavalues correspond to an inoperative sensor. In some embodiments, thepredictive model is a classification model, wherein the future datavalues indicate a predicted future state of a system comprising the oneor more sensor devices. In some embodiments, the sensors are RFIDsensors associated with cargo, wherein the future data values indicatefuture locations of the cargo. In some embodiments, the sensors aresecurity cameras, wherein the data stream comprises motion vectorsextracted from video data captured by the security cameras. In someembodiments, the sensors are vibration sensors measuring vibrationsgenerated by machines, wherein the future data values indicate apotential need for maintenance of the machines.

According to some embodiments of the present disclosure, a digitalproduct network system is disclosed. The system includes a set ofdigital products each having a product processor, a product memory, anda product network interface. The system further includes a productnetwork control tower having a control tower processor, a control towermemory, and a control tower network interface. The product processor andthe control tower processor collectively include non-transitoryinstructions that program the digital product network system to:generate product level data at the product processor; transmit theproduct level data from the product network interface; receive theproduct level data at the control tower network interface; encode theproduct level data as a product level data structure configured toconvey parameters indicated by the product level data across the set ofdigital products; and write the product level data structure to at leastone of the product memory and the control memory.

In some embodiments, the product network control tower is at least oneof a remotely located server or at least one control product of the setof digital products. In some embodiments, the product processor and thecontrol tower processor are further programmed to communicate based on ashared communication system configured for facilitating communication ofthe product level data from the set of digital products amongstthemselves and with the product control tower. In some embodiments, theset of digital products and the product network control tower have a setof microservices and a microservices architecture. In some embodiments,the system further includes a display associated with at least one ofthe product network control tower or the set of digital products,wherein the digital product network system is further programmed to:generate a graphical user interface with at least one user interfacedisplay; generate the parameters of at least one digitally enabledproduct of the set of digital products in the at least one userinterface display; and generate a proximity display of proximal digitalproducts of the set of digital products in the at least one userinterface display.

In some embodiments, generating the proximity display includesgenerating the proximity display of proximal products that aregeographically proximate. In some of these embodiments, the digitalproduct network is further programmed to filter the proximal products byat least one of product type, product capability, or product brand.Additionally or alternatively, generating the proximity display includesgenerating the proximity display of proximal products that are proximateto one of the set of digital products by product type proximity, productcapability proximity, or product brand proximity.

In some embodiments, the digital product network system is furtherprogrammed to define a data integration system. In some embodiments, thedigital product network system is further programmed for providing edgecomputation and edge intelligence configured for edge distributeddecision making among the set of digital products. In some embodiments,the digital product network system is further programmed for providingedge computation and edge intelligence configured for edge networkbandwidth management between or out of the set of digital products.

In some embodiments, the digital product network system is furtherprogrammed to have a distributed ledger system. In some of theseembodiments, the distributed ledger system wherein is a Block chainledger. In some embodiments, the digital product network system isfurther programmed to have a quality management system having a systemfor capturing product complaints at the set of digital products. In someembodiments, the digital product network system is further programmedfor: identifying a condition of the set of digital products; encodingthe condition as one of the parameters of the product level datastructure; and at least one of tracking or monitoring the conditionacross the set of digital products.

In some embodiments, the digital product network system is furtherprogrammed to have a smart contract system for enabling the creation ofsmart contracts based on the product level data structure. In some ofthese embodiments, the digital product network system is furtherprogrammed for configuring the smart contracts based on aco-location-sensitive configuration of terms such that smart contractterms and conditions depend on proximity of a plurality of digitalproducts of the set of digital products. In some embodiments, thedigital product network system is further programmed to have a roboticprocess automation (RPA) system configured to gamify an interactionbased on what digital products are in the set of digital products. Insome embodiments, the digital product network system is furtherprogrammed to have a robotic process automation (RPA) system and togenerate RPA processes based on use of a plurality of digital productsof the set of digital products.

According to some embodiments of the present disclosure, a computerizedmethod for a processor that is at least one of a set of digital productsor a product network control tower, the set of digital products eachhaving a product processor, a product memory, and a product networkinterface, the product network control tower having a control towerprocessor, a control tower memory, and a control tower network interfaceis disclosed. The method includes generating product level data at theproduct processor. The method further includes transmitting the productlevel data from the product network interface. The method furtherincludes receiving the product level data at the control tower networkinterface. The method further includes encoding the product level dataas a product level data structure configured to convey parametersindicated by the product level data across the set of digital products.The method further includes writing the product level data structure toat least one of the product memory and the control memory.

According to some embodiments of the present disclosure, a digitalproduct network system is disclosed. The system includes a set ofdigital products each having a product memory, a product networkinterface, and a product processor programmed with product instructions.The system further includes a product network control tower having acontrol tower memory, a control tower network interface, and a controltower processor programmed with control tower instructions. The systemfurther includes a digital twin system defined at least in part by atleast one of the product instructions or the control tower instructionsto encode a set of digital twins representing the set of digitalproducts.

In some embodiments, the digital twin system is further defined toencode hierarchical digital twins. In some embodiments, the digital twinsystem is further defined to encode a set of composite digital twinseach made up of a set of discrete digital twins of the set of digitalproducts. In some embodiments, the digital twin system is furtherdefined to encode a set of digital product digital twins representing aplurality of digital products of the set of digital products. In someembodiments, the digital twin system is further defined to model trafficof moving elements in the set of digital products. In some embodiments,the digital twin system is further defined to have a playback interfacefor the set of digital twins wherein a user may replay data for asituation in the digital twin and observe visual representations ofevents related to the situation.

In some embodiments, the digital twin system is further defined to:generate an adaptive user interface; and adapt for the adaptive userinterface at least one of available data, features, or visualrepresentations based on at least one of a user's association with orproximity to digital products of the set of digital products. In someembodiments, the digital twin system is further defined to manageinteractions among multiple digital product digital twins of the set ofdigital twins. In some embodiments, the digital twin system is furtherdefined to generate and update a self-expanding digital twin associatedwith the set of digital products.

In some embodiments, the digital twin system is further defined to:aggregate performance data from a plurality of digital twins of the setof digital twins about a common asset type represented in the pluralityof digital twins; and associate the aggregated performance data as aperformance data set for retrieval. In some embodiments, the digitaltwin system is further defined to match owners of identical or similarproducts in a market for digital twin data. In some embodiments, thedigital twin system is further defined to lock the set of digital twinsupon detection of a security threat in a digital product of the set ofdigital products.

In some embodiments, the digital twin system is further defined to havean in-twin marketplace. In some of these embodiments, the in-twinmarketplace offers data. In some embodiments, the in-twin marketplaceoffers services. In some embodiments, the digital twin system is furtherdefined to offer components. In some embodiments, the digital twinsystem is further defined to include application program interfaces(APIs) between the set of digital twins and marketplaces related to theset of digital products. In some embodiments, the digital twin system isfurther defined to have a twin store market system for providing atleast one of access or rights to at least one of the set of digitaltwins or data associated with the set of digital twins.

According to some embodiments of the present disclosure, a computerizedmethod for a processor that is at least one of a set of digital productsor a product network control tower, the set of digital products eachhaving a product processor, a product memory, and a product networkinterface, the product network control tower having a control towerprocessor, a control tower memory, and a control tower network interfaceis disclosed. The method includes defining a digital twin system at theprocessor. The method further includes encoding a set of digital twinsin the digital twin system, the set of digital twins representing theset of digital products.

In some embodiments, the method further includes encoding a set ofcomposite digital twins each made up of a set of discrete digital twinsof the set of digital products.

According to some embodiments of the present disclosure, a method forexecuting a quantum computing task is disclosed. The method includesproviding a quantum computing system. The method further includesreceiving a request, from a quantum computing client, to execute aquantum computing task via the quantum computing system. The methodfurther includes executing the requested quantum computing task via thequantum computing system. The method further includes returning aresponse related to the executed quantum computing task to the quantumcomputing client.

In some embodiments, the quantum computing system is a quantum annealingcomputing system. In some embodiments, the quantum computing systemsupports one or more quantum computing models selected from the set of:quantum circuit model, the quantum Turing machine, spintronic computingsystem, adiabatic quantum computing system, one-way quantum computer,and quantum cellular automata.

In some embodiments, the quantum computing system is physicallyimplemented using an analog approach. In some of these embodiments, theanalog approaches may be selected from the list of: quantum simulation,quantum annealing, and adiabatic quantum computation. In someembodiments, the quantum computing system is physically implementedusing a digital approach. In some embodiments, the quantum computingsystem is an error-corrected quantum computer. In some embodiments, thequantum computing system applies trapped ions to execute the quantumcomputing task.

In some embodiments, the quantum computing task relates to automaticallydiscovering smart contract configuration opportunities in a value chainnetwork. In some of these embodiments, the quantum-established smartcontract applications are selected from the set of: booking a set ofrobots from a robotic fleet, booking a smart container from a smartcontainer fleet, and executing transfer pricing agreements betweensubsidiaries. In some embodiments, the quantum computing task relates torisk identification or risk mitigation. In some embodiments, the quantumcomputing task relates to accelerated sampling from stochastic processesfor risk analysis. In some embodiments, the quantum computing taskrelates to graph clustering analysis for anomaly or fraud detection. Insome embodiments, the quantum computing task relates to generating aprediction.

According to some embodiments of the present disclosure, a method forexecuting a quantum computing optimization task is disclosed. The methodincludes providing a quantum computing system. The method furtherincludes receiving a request, from a quantum computing client, toexecute a quantum computing optimization task via the quantum computingsystem. The method further includes executing the requested quantumcomputing optimization task via the quantum computing system. The methodfurther includes returning a response related to the executed quantumcomputing optimization task to the quantum computing client.

In some embodiments, the quantum computing system is a quantum annealingcomputing system. In some embodiments, the quantum computing system is aquantum annealing computing system. In some embodiments, the quantumcomputing system supports one or more quantum computing models selectedfrom the set of: quantum circuit model, the quantum Turing machine,spintronic computing system, adiabatic quantum computing system, one-wayquantum computer, and quantum cellular automata.

In some embodiments, the quantum computing system is physicallyimplemented using an analog approach. In some of these embodiments, theanalog approaches may be selected from the list of: quantum simulation,quantum annealing, and adiabatic quantum computation. In someembodiments, the quantum computing system is physically implementedusing a digital approach. In some embodiments, the quantum computingsystem is an error-corrected quantum computer. In some embodiments, thequantum computing system applies trapped ions to execute the quantumcomputing task.

In some embodiments, the quantum computing optimization task is a smartcontainer-based freight transportation price optimization task. In someof these embodiments, the quantum computing system is configured to useq-bit-based computational methods to optimize pricing. In someembodiments, the quantum computing system is configured to optimize thedesign or configuration of a product, device, vehicle, or service in avalue chain network.

According to some embodiments of the present disclosure, a smartshipping container system is disclosed. The system includes a shippingcontainer housing. The system further includes an artificialintelligence-enabled chipset.

In some embodiments, the smart shipping container system type isselected from the set of: tank container, general-purpose dry van,rolling floor container, garmentainer, ventilated container,temperature-controlled container, bulk container, open-top container,open-side container, log cradle, platform-based container, rotatingcontainer, mixing container, aviation container, automotive container,and bioprotective container. In some embodiments, the smart shippingcontainer system is a smart package. In some embodiments, the smartshipping container system includes a mechanism to enable expanding orretracting external or internal walls, housing elements, or otherinternal elements, such as to increase or decrease the volume of thecontainer or to vary the dimensions of one or more partitions of thespace within the container. In some embodiments, the smart shippingcontainer system includes a self-assembling mechanism. In someembodiments, the smart shipping container system includes aself-disassembling mechanism. In some embodiments, the smart shippingcontainer shape is selected from the set of: rectangular, cube, sphere,cylindrical, organic-like, and biometric. In some embodiments, the smartshipping container material, at least in part, is selected from the setof: corrugated weathering steel, steel alloys, stainless steel,aluminum, cast iron, concrete, ceramic material(s), other alloys, glass,other metals, plastics, plywood, bamboo, cardboard, and wood. In someembodiments, the smart shipping container system is a 3D-printed smartcontainers. In some embodiments, the smart shipping container systemincludes a 3D printer.

According to some embodiments of the present disclosure, a smartshipping container system is disclosed. The system includes a shippingcontainer housing. The system further includes an artificialintelligence-enabled chipset. The shipping container is configured to beself-driving.

In some embodiments, the smart shipping container system type isselected from the set of: tank container, general-purpose dry van,rolling floor container, garmentainer, ventilated container,temperature-controlled container, bulk container, open-top container,open-side container, log cradle, platform-based container, rotatingcontainer, mixing container, aviation container, automotive container,and bioprotective container. In some embodiments, the smart shippingcontainer system is a smart package. In some embodiments, the smartshipping container system includes a mechanism to enable expanding orretracting external or internal walls, housing elements, or otherinternal elements, such as to increase or decrease the volume of thecontainer or to vary the dimensions of one or more partitions of thespace within the container. In some embodiments, the smart shippingcontainer system includes a self-assembling mechanism. In someembodiments, the smart shipping container system includes aself-disassembling mechanism. In some embodiments, the smart shippingcontainer shape is selected from the set of: rectangular, cube, sphere,cylindrical, organic-like, and biometric. In some embodiments, the smartshipping container material, at least in part, is selected from the setof: corrugated weathering steel, steel alloys, stainless steel,aluminum, cast iron, concrete, ceramic material(s), other alloys, glass,other metals, plastics, plywood, bamboo, cardboard, and wood. In someembodiments, the smart shipping container system is a 3D-printed smartcontainers. In some embodiments, the smart shipping container systemincludes a 3D printer.

According to some embodiments of the present disclosure, a method forupdating one or more properties of one or more shipping digital twins isdisclosed. The method includes receiving a request to update one or moreproperties of one or more shipping digital twins. The method furtherincludes retrieving the one or more shipping digital twins required tofulfill the request. The method further includes retrieving one or moredynamic models required to fulfill the request. The method furtherincludes selecting data sources from a set of available data sourcesbased on the one or more inputs of the one or more dynamic models. Themethod further includes retrieving data from selected data sources. Themethod further includes calculating one or more outputs using theretrieved data as one or more inputs to the one or more dynamic models.The method further includes updating one or more properties of the oneor more shipping digital twins based on the output of the one or moredynamic models.

In some embodiments, the digital twins are digital twins of smartcontainers. In some embodiments, the digital twins are digital twins ofshipping environments. In some embodiments, the digital twins aredigital twins of shipping entities. In some embodiments, the dynamicmodels take data selected from the set of vibration, temperature,pressure, humidity, wind, rainfall, tide, storm surge, cloud cover,snowfall, visibility, radiation, audio, video, image, water level,quantum, flow rate, signal power, signal frequency, motion,displacement, velocity, acceleration, lighting level, financial, cost,stock market, news, social media, revenue, worker, maintenance,productivity, asset performance, worker performance, worker responsetime, analyte concentration, biological compound concentration, metalconcentration, and organic compound concentration data.

In some embodiments, the data source is selected from the set of anInternet of Things connected device, a machine vision system, an analogvibration sensor, a digital vibration sensor, a fixed digital vibrationsensor, a tri-axial vibration sensor, a single axis vibration sensor, anoptical vibration sensor, and a crosspoint switch. In some embodiments,retrieving the one or more dynamic models includes identifying the oneor more dynamic models based on the one or more properties indicated inthe request and a respective type of the one or more digital twins. Insome embodiments, the one or more dynamic models are identified using alookup table.

According to some embodiments of the present disclosure, a robot fleetmanagement platform is disclosed. The platform includes acomputer-readable storage system that stores a resources data store thatmaintains: a robot inventory that indicates a plurality of robots thatcan be assigned to a robot fleet, and for each respective robot, a setof baseline features of the robot and a respective status of the robot,wherein the robot inventory of robots includes a plurality ofmulti-purpose robots that can be configured for different tasks anddifferent environments; and a components inventory that indicatesdifferent components that can be provisioned to one or moremulti-purpose robots, and for each component, a respective set ofextended capabilities corresponding to the component and a respectivestatus of the component. The platform further includes a set of one ormore processors that execute a set of computer-readable instructions.The set of one or more processors collectively receive a request for arobotic fleet to perform a job. The set of one or more processorscollectively determine a job definition data structure based on therequest, the job definition data structure defining a set of tasks thatare to be performed in performance of the job. The set of one or moreprocessors collectively determine a robot fleet configuration datastructure corresponding to the job based on the set of tasks and therobot inventory, wherein the robot fleet configuration data structureassigns a plurality of robots selected from the robot inventory to theset of tasks defined in the job definition data structure and theplurality of robots includes one or more assigned multi-purpose robots.The set of one or more processors collectively determine a respectiveconfiguration for each respective assigned multi-purpose robot based onthe respective task that is assigned to the assigned multi-purpose robotand the components inventory. The set of one or more processorscollectively configure the one or more assigned multi-purpose robotsbased on the respective configurations. The set of one or moreprocessors collectively deploy the robotic fleet to perform the job.

In some embodiments, the robot inventory includes special purposerobots. In some embodiments, determining the robot fleet configurationdata structure is further based on an environment of the job. In someembodiments, determining the robot fleet configuration data structure isfurther based on a budget for the job. In some embodiments, determiningthe robot fleet configuration data structure is further based on atimeline for completing the job. In some embodiments, the robotinventory includes special purpose robots and to determine the robotfleet configuration data structure is further based on an availableinventory of the special purpose robots. In some embodiments,determining a respective configuration for each respective assignedmulti-purpose robot is further based on an environment of the job. Insome embodiments, determining a respective configuration for eachrespective assigned multi-purpose robot is further based on a budget forthe job. In some embodiments, determining a respective configuration foreach respective assigned multi-purpose robot is further based on atimeline for completing the job. In some embodiments, configuring theone or more assigned multi-purpose robots includes configuring at leastone robot system selected from a list of robot systems including a robotbaseline system, a module system, a robot control system, and a robotsecurity system.

In some embodiments, configuring the one or more assigned multi-purposerobots includes configuring one or more of a software robot module or ahardware robot module. In some of these embodiments, the hardware robotmodule is an interchangeable module.

In some embodiments, configuring the one or more assigned multi-purposerobots task includes accessing a robot module system via at least one ofa physical interface module and a control interface module. In someembodiments, configuring the one or more assigned multi-purpose robotsincludes configuring one or more modules of a robot baseline system, theone or more modules selected from a baseline module list including anenergy storage and power distribution system, an electromechanical andelectro-fluidic system, a transport system, and a vision and sensingsystem. In some embodiments, configuring the one or more assignedmulti-purpose robots includes configuring a 3D printing system toproduce at least one hardware robot module.

In some embodiments, configuring the one or more assigned multi-purposerobots is based on one or more characteristics of a target operatingenvironment. In some of these embodiments, a target operatingenvironment is one or more of land-based, sea-based, submerged,in-flight, subterranean, and below-freezing ambient temperature.

In some embodiments, configuring the one or more assigned multi-purposerobots includes configuring an energy storage and power distributionsystem to utilize two or more distinct power sources based on an aspectof one of a task and an operating environment. In some of theseembodiments, a first distinct power source of the two or more distinctpower sources is a mobile power source of the multi-purpose robot and asecond distinct power source of the two or more distinct power sourcesis a fixed position power source that provides power to the robot via awireless power signal.

In some embodiments, configuring the one or more assigned multi-purposerobots includes configuring a propulsion system of the robot toadaptably utilize one or more legs for locomotion. In some embodiments,configuring the one or more assigned multi-purpose robots includesprovisioning one or more modules identified in a job execution plan tothe multi-purpose robot. In some of these embodiments, the one or moremodules is a hardware module. Additionally or alternatively, the one ormore modules is a software module.

In some embodiments, configuring the one or more assigned multi-purposerobots includes provisioning one or more of appendages, sensor sets,chipsets, and motive adaptors to the multi-purpose robot based on atleast one task in a set of target tasks for the robot that areidentified in a job execution plan. In some embodiments, configuring theone or more assigned multi-purpose robots includes analyzing a jobexecution plan that defines a fleet of robots and configuring at leastone multi-purpose robot of the fleet of robots. In some embodiments,configuring the one or more assigned multi-purpose robots includesprovisioning a local manager capability that enables the multi-purposerobot to control one or more robots.

According to some embodiments of the present disclosure, a method ofconfiguring a multi-purpose robot of a fleet of robots is disclosed. Themethod includes receiving a request for a robotic fleet to perform ajob. The method further includes defining a set of tasks that are to beperformed in performance of the job. The method further includesassigning a plurality of robots selected from a robot inventory to theset of tasks based on the set of tasks and a robot inventory datastructure that indicates a plurality of robots that can be assigned to arobot fleet, and for each respective robot, a set of baseline featuresof the robot and a respective status of the robot, wherein the pluralityof robots includes one or more assigned multi-purpose robots that can beconfigured for different tasks and different environments. The methodfurther includes determining a respective configuration for eachrespective assigned multi-purpose robot based on the respective taskthat is assigned to the assigned multi-purpose robot and a componentsinventory that indicates different components that can be provisioned toone or more multi-purpose robots, and for each component, a respectiveset of extended capabilities corresponding to the component and arespective status of the component. The method further includesconfiguring the one or more assigned multi-purpose robots based on therespective configurations. The method further includes deploying therobotic fleet to perform the job.

In some embodiments, the robot inventory includes special purposerobots. In some embodiments, assigning a plurality of robots selectedfrom the robot inventory is further based on an environment of the job.In some embodiments, assigning a plurality of robots selected from therobot inventory is further based on a budget for the job. In someembodiments, assigning a plurality of robots selected from the robotinventory is further based on a timeline for completing the job. In someembodiments, the robot inventory includes special purpose robots and toassigning a plurality of robots selected from the robot inventory isfurther based on an available inventory of the special purpose robots.In some embodiments, determining a respective configuration for eachrespective assigned multi-purpose robot is further based on anenvironment of the job. In some embodiments, determining a respectiveconfiguration for each respective assigned multi-purpose robot isfurther based on a budget for the job. In some embodiments, determininga respective configuration for each respective assigned multi-purposerobot is further based on a timeline for completing the job. In someembodiments, configuring the one or more assigned multi-purpose robotsincludes configuring at least one robot system selected from a list ofrobot systems including a robot baseline system, a module system, arobot control system, and a robot security system. In some embodiments,configuring the one or more assigned multi-purpose robots includesconfiguring one or more of a software robot module or a hardware robotmodule. In some embodiments, configuring the one or more assignedmulti-purpose robots task includes accessing a robot module system viaat least one of a physical interface module and a control interfacemodule. In some embodiments, configuring the one or more assignedmulti-purpose robots includes configuring one or more modules of a robotbaseline system, the one or more modules selected from a baseline modulelist including an energy storage and power distribution system, anelectromechanical and electro-fluidic system, a transport system, and avision and sensing system. In some embodiments, configuring the one ormore assigned multi-purpose robots is based on one or morecharacteristics of a target operating environment.

In some embodiments, configuring the one or more assigned multi-purposerobots includes configuring an energy storage and power distributionsystem to utilize two or more distinct power sources based on an aspectof one of a task and an operating environment. In some of theseembodiments, a first distinct power source of the two or more distinctpower sources is a mobile power source of the multi-purpose robot and asecond distinct power source of the two or more distinct power sourcesis a fixed position power source that provides power to the robot via awireless power signal.

In some embodiments, configuring the one or more assigned multi-purposerobots includes configuring a propulsion system of the robot toadaptably utilize one or more legs for locomotion. In some embodiments,configuring the one or more assigned multi-purpose robots includesprovisioning one or more modules identified in a job execution plan tothe multi-purpose robot. In some embodiments, configuring the one ormore assigned multi-purpose robots includes provisioning one or more ofappendages, sensor sets, chipsets, and motive adaptors to themulti-purpose robot based on at least one task in a set of target tasksfor the robot that are identified in a job execution plan. In someembodiments, configuring the one or more assigned multi-purpose robotsincludes analyzing a job execution plan that defines a fleet of robotsand configuring at least one multi-purpose robot of the fleet of robots.In some embodiments, configuring the one or more assigned multi-purposerobots includes provisioning a local manager capability that enables themulti-purpose robot to control one or more robots.

According to some embodiments of the present disclosure, a robotic fleetmanagement platform is disclosed. The platform includes acomputer-readable storage system that stores a resources data store thatmaintains a fleet resource inventory that indicates a plurality of fleetresources that can be assigned to a robotic fleet, and for eachrespective fleet resource, maintenance status data including amaintenance history, a predicted maintenance need, and a preventivemaintenance schedule; and a maintenance management library of fleetresource maintenance requirements that facilitates determiningmaintenance workflows, service actions, and service parts for at leastone fleet resource of the plurality of fleet resources indicated in thefleet resource inventory. The platform further includes a set of one ormore processors that execute a set of computer-readable instructions.The set of one or more processors collectively calculate the predictedmaintenance need of a fleet resource based on anticipated component wearand anticipated component failure of one or more components of the atleast one fleet resource, wherein the anticipated component wear andanticipated component failure of the one or more components is derivedfrom machine learning-based analysis of the maintenance status data inthe fleet resource inventory. The set of one or more processorscollectively monitor a health state of the fleet resource, wherein thehealth state is determined from sensor data received from the fleetresource. The set of one or more processors collectively adapt thepreventive maintenance schedule for the fleet resource by indicating anew preventive maintenance schedule for at least one item of maintenancefor the fleet resource based on the predicted maintenance need, thehealth state, and the fleet resource maintenance requirements of thefleet resource. The set of one or more processors collectively initiatea service action of the at least one item of maintenance for the fleetresource based on the fleet resource maintenance requirements and thenew preventive maintenance schedule.

In some embodiments, the set of one or more processors further predictfleet resource maintenance needs based on digital twin-based simulationof a digital twin of the at least one fleet resource. In someembodiments, the at least one fleet resource is a robotic operatingunit.

In some embodiments, a predictive maintenance intelligence service layerpredicts at least one of the anticipated component wear or theanticipated component failure by applying a clustering algorithm toidentify at least one failure pattern in a set of failure data. In someof these embodiments, the predictive maintenance intelligence servicelayer correlates patterns of failure to wear-down behavior present incurrent operational data thereby producing a pre-failure maintenanceplan. In some of these embodiments, the predictive maintenanceintelligence service layer adjusts a preventive maintenance plan for arobotic fleet resource based on the correlated patterns of failure forsimilar types of robotic fleet resources. Additionally or alternatively,the predictive maintenance intelligence service layer predicts fleetresource maintenance needs based on digital twin-based simulation of adigital twin of at least one fleet resource.

In some embodiments, adapting the preventive maintenance scheduleincludes interacting with a fleet configuration system by sharingjob-impacting fleet resource maintenance knowledge. In some embodiments,causing a service action includes configuring a set of 3D printingrequirements for facilitating field maintenance of a fleet resource. Insome of these embodiments, the 3D printing requirements are configuredbased on a predicted maintenance activity for the fleet resource. Insome embodiments, the new preventive maintenance schedule includesscheduled field maintenance of at least one fleet resource.

In some embodiments, the new preventive maintenance schedule includesscheduled repair depot-based maintenance of at least one fleet resource.In some of these embodiments, the at least one fleet resource is a smartcontainer operating unit. Additionally or alternatively, the at leastone fleet resource is a robotic operating unit. In some embodiments, theplatform further includes a mobile maintenance vehicle. In someembodiments, the platform further includes a repair depot. In someembodiments, the platform further includes a third-party maintenanceservice provider. In some embodiments, adapting the preventivemaintenance schedule includes adapting a maintenance schedule for atleast one inactive fleet resource based on an evaluation of amaintenance need for the at least one inactive fleet resource.

In some embodiments, the set of one or more processors further monitor astate of at least one fleet resource by monitoring communications of theat least one fleet resource for an indication of a maintenance need. Insome of these embodiments, the at least one fleet resource is a roboticoperating unit. Additionally or alternatively, the indication of amaintenance need includes a lack of a heartbeat signal to a fleetresource health monitor resource. Additionally or alternatively, themaintenance need of the at least one fleet resource includes a potentialservice condition. In some of these embodiments, the potential servicecondition includes one or more of reduced power output, exposure toexcess ambient conditions, or a leak.

In some embodiments, the set of one or more processors further deployssoftware-based maintenance monitoring probes to operating or supervisorysoftware of the at least one fleet resource. In some of theseembodiments, the probes monitor information in a data store of the atleast one fleet resource that stores operating state information.Additionally or alternatively, the probes activate self-test operatingmodes of the at least one fleet resource. Additionally or alternatively,the probes collect data that provides indications of maintenance needsof the at least one fleet resource.

In some embodiments, the set of one or more processors further deploysone or more maintenance fleet resources within one or more smartcontainers. In some embodiments, adapting the preventive maintenanceschedule includes adapting a maintenance schedule for at least one fleetresource based on operator input regarding a state of the at least onefleet resource. In some embodiments, causing a service action includesautomation of maintenance activities for the at least one fleetresource. In some embodiments, adapting the preventive maintenanceschedule includes adapting a maintenance schedule for the at least onefleet resource based on artificial intelligence-based prediction ofmaintenance instances.

In some embodiments, adapting the preventive maintenance scheduleincludes adapting a maintenance schedule for the at least one fleetresource based on a machine learning system that identifies newopportunities for scheduling and performing maintenance. In some ofthese embodiments, the machine learning system analyzes performance datafor the at least one other robot that has been maintained for operationin certain conditions. In some of these embodiments, a cooling system ofthe other robot has been maintained prior to operating in a hightemperature environment and the performance data reflects operation ofthe at least one other robot in the certain conditions.

In some embodiments, adapting the preventive maintenance scheduleincludes adapting a maintenance schedule for the at least one fleetresource based on one or more of: maintenance rules established for ateam, maintenance rules established for a fleet, maintenance rulesestablished by a shipper, maintenance rules determined by a regulatoryagency. In some embodiments, adapting the preventive maintenanceschedule includes determining one or more of maintenance workflows,service actions, or needed parts for maintaining the at least one fleetresource based on one or more of association tables, data sets,databases, or maintenance management libraries. In some embodiments,causing a service action includes assigning a maintenance activity to afleet resource selected from a list of fleet resources including amaintenance smart container, a human technician, and a third-partyservice provider. In some embodiments, causing a service action includesdeploying a maintenance service that performs maintenance of the atleast one fleet resource via a set of self-maintenance protocols for atleast one of self-cleaning and calibrating end effector operations. Insome embodiments, causing a service action includes interacting with afleet configuration system responsive to an indication of a compromisedcapability of the at last one robot, the interaction resulting in achange in assignment of the at least one fleet resource based on thecompromised capability. In some embodiments, causing a service action isbased on an interaction with a digital twin of the at least one fleetresource being operated by a fleet intelligence service that predicts amaintenance need of the at least one fleet resource. In someembodiments, causing a service action includes coordinating maintenanceactivities with job scheduling to ensure that preventable interruptionsdue to lack of maintenance are prevented.

According to some embodiments of the present disclosure, a robotic fleetresource provisioning system is disclosed. The system includes acomputer-readable storage system that stores: a fleet resources datastore that maintains a fleet resource inventory that indicates aplurality of fleet resources that can be provisioned as a set of fleetresources, and for each respective fleet resource, a set of features ofthe resource, configuration requirements of the resource, and arespective status of the resource; and a set of resource provisioningrules that are accessible to an intelligence layer to ensure thatprovisioned resources comply with the provisioning rules. The systemfurther includes a set of one or more processors that execute a set ofcomputer-readable instructions. The set of one or more processorscollectively receive a request for a robotic fleet to perform a job. Theset of one or more processors collectively determine a job definitiondata structure based on the request, the job definition data structuredefining a set of tasks that are to be performed in performance of thejob. The set of one or more processors collectively determine a roboticfleet configuration data structure corresponding to the job based on theset of tasks and the fleet resource inventory, wherein the robotic fleetconfiguration data structure assigns a plurality of resources selectedfrom the fleet resource inventory to the set of tasks defined in the jobdefinition data structure. The set of one or more processorscollectively determine a respective provisioning configuration for eachrespective fleet resource based on the respective task to which thefleet resource is assigned, the set of features of the fleet resource,the configuration requirements of the fleet resource, and the respectivestatus of the fleet resource. The set of one or more processorscollectively provision the respective fleet resource based on therespective provisioning configuration and the provisioning rules. Theset of one or more processors collectively deploy the robotic fleet toperform the job.

In some embodiments, the respective status of the resource includes ageneral availability of the resource. In some embodiments, determiningthe robotic fleet configuration data structure is further based on anenvironment of the job. In some embodiments, determining the roboticfleet configuration data structure is further based on a budget for thejob. In some embodiments, determining the robotic fleet configurationdata structure is further based on a timeline for completing the job. Insome embodiments, the fleet resource inventory includes one or moretypes of robots and to determine the robotic fleet configuration datastructure is further based on an available inventory of the one or moretypes of robots. In some embodiments, determining a provisioningconfiguration for each respective fleet resource is further based on anenvironment of the job. In some embodiments, determining a provisioningconfiguration for each respective fleet resource is further based on abudget for the job. In some embodiments, determining a provisioningconfiguration for each respective assigned fleet resource is furtherbased on a timeline for completing the job. In some embodiments, thefleet resource inventory includes computing resources selected from alist of computing resources comprising on-robot computing resources,robot operating unit-local fleet-controlled computing resources, cloudbased computing resources, computing modules, or computing chips.

In some embodiments, provisioning the respective fleet resource includesprovisioning one or more of a software robot module or a hardware robotmodule. In some of these embodiments, the hardware robot module is aninterchangeable module.

In some embodiments, the fleet resource inventory includes a pluralityof digital resources. In some of these embodiments, provisioning arespective one of the plurality of digital resources includes one ormore of software update pushing, resource access credentialing, or fleetresource data storage configuration, allocation, or utilization. In someembodiments, provisioning a respective fleet resource includesprovisioning a consumable resource sourced from at least one of aspecialized supply chain, a job requestor resource supply, afleet-specific stockpile, a job-specific stockpile, or a fleetteam-specific stockpile.

In some embodiments, provisioning the respective fleet resource is basedon one or more characteristics of a target operating environment. Insome of these embodiments, a target operating environment is one or moreof land-based, sea-based, submerged, in-flight, subterranean, andbelow-freezing ambient temperature.

In some embodiments, provisioning the respective fleet resource includes3D printing the respective resource for provisioning. In someembodiments, provisioning the respective fleet resource is based onterms of a smart contract that constrains provisioning of fleetresources. In some embodiments, the fleet resource inventory includesplatform resources and to provision the respective fleet resourceincludes provisioning at least one platform resource selected from alist of platform resources including computing resources, a fleetconfiguration system, a platform intelligence layer, a platform dataprocessing system, and a fleet security system. In some of theseembodiments, determining a robotic fleet configuration data structure isfurther based on a negotiated charge for provisioning a platformresource. Additionally or alternatively, determining a robotic fleetconfiguration data structure includes a negotiation workflow foracceptance of the job request.

In some embodiments, provisioning the respective fleet resource includesprovisioning one or more fleet resources identified in a job executionplan. In some embodiments, provisioning the respective fleet resourceincludes provisioning one or more of appendages, sensor sets, chipsets,and motive adaptors to a robot based on at least one task in a set oftarget tasks for the robot that are identified in a job execution plan.In some embodiments, provisioning the respective fleet resource includesanalyzing a job execution plan that defines resources for a fleet ofrobots for performing at least one task. In some embodiments, the set ofone or more processors execute the set of computer-readable instructionscooperatively with at least one of a fleet configuration system, a fleetresource scheduling system, a fleet security system, and a fleetutilization system.

According to some embodiments of the present disclosure, a method ofprovisioning robotic fleet resources is disclosed. The method includesreceiving a request for a robotic fleet to perform a job. The methodfurther includes determining a job definition data structure based onthe request, the job definition data structure defining a set of tasksthat are to be performed in performance of the job. The method furtherincludes determining a robotic fleet configuration data structurecorresponding to the job based on the set of tasks and a fleet resourceinventory that indicates a plurality of fleet resources, and for eachrespective fleet resource, a set of features of the resource,configuration requirements of the resource, and a respective status ofthe resource, wherein the robotic fleet configuration data structureassigns a plurality of resources selected from the fleet resourceinventory to the set of tasks defined in the job definition datastructure. The method further includes determining a respectiveprovisioning configuration for each respective fleet resource based onthe respective task to which the fleet resource is assigned, the set offeatures of the fleet resource, the configuration requirements of thefleet resource, and the respective status of the fleet resource. Themethod further includes provisioning the respective fleet resource basedon the respective provisioning configuration and a set of resourceprovisioning rules that are accessible to an intelligence layer toensure that provisioned resources comply with the provisioning rules.The method further includes deploying the robotic fleet to perform thejob.

In some embodiments, the respective status of the resource includes ageneral availability of the resource. In some embodiments, determiningthe robotic fleet configuration data structure is further based on anenvironment of the job. In some embodiments, determining the roboticfleet configuration data structure is further based on a budget for thejob. In some embodiments, determining the robotic fleet configurationdata structure is further based on a timeline for completing the job. Insome embodiments, the fleet resource inventory includes one or moretypes of robots and determining the robotic fleet configuration datastructure is further based on an available inventory of the one or moretypes of robots. In some embodiments, determining a provisioningconfiguration for each respective fleet resource is further based on anenvironment of the job. In some embodiments, determining a provisioningconfiguration for each respective fleet resource is further based on abudget for the job. In some embodiments, determining a provisioningconfiguration for each respective assigned fleet resource is furtherbased on a timeline for completing the job. In some embodiments, thefleet resource inventory includes computing resources selected from alist of computing resources comprising on-robot computing resources,robot operating unit-local fleet-controlled computing resources, cloudbased computing resources, computing modules, or computing chips.

In some embodiments, provisioning the respective fleet resource includesprovisioning one or more of a software robot module or a hardware robotmodule. In some of these embodiments, the hardware robot module is aninterchangeable module.

In some embodiments, the fleet resource inventory includes a pluralityof digital resources. In some of these embodiments, provisioning arespective one of the plurality of digital resources includes one ormore of software update pushing, resource access credentialing, or fleetresource data storage configuration, allocation, or utilization.

In some embodiments, provisioning a respective fleet resource includesprovisioning a consumable resource sourced from at least one of aspecialized supply chain, a job requestor resource supply, afleet-specific stockpile, a job-specific stockpile, or a fleetteam-specific stockpile. In some embodiments, provisioning therespective fleet resource is based on one or more characteristics of atarget operating environment. In some of these embodiments, a targetoperating environment is one or more of land-based, sea-based,submerged, in-flight, subterranean, and below-freezing ambienttemperature. In some embodiments, provisioning the respective fleetresource includes 3D printing the respective resource for provisioning.In some embodiments, provisioning the respective fleet resource is basedon terms of a smart contract that constrains provisioning of fleetresources.

In some embodiments, the fleet resource inventory includes platformresources and provisioning the respective fleet resource includesprovisioning at least one platform resource selected from a list ofplatform resources including computing resources, a fleet configurationsystem, a platform intelligence layer, a platform data processingsystem, and a fleet security system. In some of these embodiments,determining a robotic fleet configuration data structure is furtherbased on a negotiated charge for provisioning a platform resource. Insome of these embodiments, determining a robotic fleet configurationdata structure includes a negotiation workflow for acceptance of the jobrequest.

In some embodiments, provisioning the respective fleet resource includesprovisioning one or more fleet resources identified in a job executionplan. In some embodiments, provisioning the respective fleet resourceincludes provisioning one or more of appendages, sensor sets, chipsets,and motive adaptors to a robot based on at least one task in a set oftarget tasks for the robot that are identified in a job execution plan.In some embodiments, provisioning the respective fleet resource includesanalyzing a job execution plan that defines resources for a fleet ofrobots for performing at least one task. In some embodiments, the methodfurther includes executing cooperatively with at least one of a fleetconfiguration system, a fleet resource scheduling system, a fleetsecurity system, and a fleet utilization system.

According to some embodiments of the present disclosure, a robotic fleetplatform for configuring robot fleets with additive manufacturingcapabilities is disclosed. The platform includes a computer-readablestorage system that stores: a fleet resources data store that maintainsa fleet resource inventory that indicates a plurality of additivemanufacturing systems that can be provisioned with a set of fleetresources, and for each respective additive manufacturing system, a setof 3D printing requirements, printing instructions that defineconfiguring an on-demand production system for 3D printing, and a statusof the additive manufacturing system; and a set of additivemanufacturing system provisioning rules that are accessible to anintelligence layer to ensure that provisioned additive manufacturingsystems comply with the provisioning rules. The platform furtherincludes a set of one or more processors that execute a set ofcomputer-readable instructions. The set of one or more processorscollectively receive a request for a robotic fleet to perform a job. Theset of one or more processors collectively determine a job definitiondata structure based on the request, the job definition data structuredefining a set of tasks that are to be performed in performance of thejob. The set of one or more processors collectively determine a roboticfleet configuration data structure corresponding to the job based on theset of tasks and the fleet resource inventory, wherein the robotic fleetconfiguration data structure assigns one or more additive manufacturingsystems selected from the fleet resource inventory to one or more of theset of tasks defined in the job definition data structure. The set ofone or more processors collectively determine a respective provisioningconfiguration for each respective additive manufacturing system based onthe respective task to which the additive manufacturing system isassigned, the set of 3D printing requirements, the printinginstructions, and the respective status of the additive manufacturingsystem. The set of one or more processors collectively provision therespective additive manufacturing system based on the respectiveprovisioning configuration and the provisioning rules. The set of one ormore processors collectively deploy the robotic fleet based on therobotic fleet configuration data structure to perform the job.

In some embodiments, provisioning the respective additive manufacturingsystem includes to provision a 3D printing capable robot. In someembodiments, the respective provisioning configuration for eachrespective additive manufacturing system includes a set of 3D printinginstructions for at least one of a job-specific end effector or anadaptor based on a context of the task to which the additivemanufacturing system is assigned. In some embodiments, the robotic fleetconfiguration data structure assigns control of at least onetransportable 3D printing additive manufacturing system to at least onerobot operating unit.

In some embodiments, determining the robotic fleet configuration datastructure is further based on availability and job site locality of 3Dprinting resources. In some of these embodiments, at least one of theavailability or job site locality of the 3D printing resource isidentified by a logistics system of the platform. In some embodiments,determining the robotic fleet configuration data structure includesassignment of at least one additive manufacturing system indicated inthe fleet resource inventory based on proximity to a job site for therequested job.

In some embodiments, determining a respective provisioning configurationfor each respective additive manufacturing system includes use of anartificial intelligence system to automate design for 3D printing of oneor more robotic accessories. In some of these embodiments, theartificial intelligence system automates design for 3D printing based oncontextual task recognition. Additionally or alternatively, theartificial intelligence system automates design for 3D printing based onautomated shape recognition capabilities. Additionally or alternatively,provisioning the respective additive manufacturing system includesprovisioning a 3D printing control capability to produce an end effectorbased on a visual and sensed analysis of an object for manipulation ofwhich the end effector is to be 3D printed.

In some embodiments, deploying the robotic fleet includes use of a fleetconfiguration scheduling resource of the platform for allocation of therespective additive manufacturing system to perform the job. In someembodiments, deploying the robotic fleet includes deploying a 3Dprinting robot to a smart container for remote, on-demand additivemanufacturing. In some embodiments, determining a respectiveprovisioning configuration for each respective additive manufacturingsystem is further based on one or more keywords of the job definitiondata structure that are indicative of an operating condition for therespective additive manufacturing system. In some embodiments, deployingthe robotic fleet includes deploying a set of autonomous 3D printingadditive manufacturing system to points of service work indicated in thejob definition data structure. In some embodiments, determining arespective provisioning configuration for each respective additivemanufacturing system includes configuring a 3D printing system toreceive a tokenized instance of a set of 3D printing instructionsassociated with a corresponding token on a distributed ledger. In someembodiments, deploying the robotic fleet includes deploying therespective additive manufacturing system as a 3D printing resourceshared among a plurality of tasks.

According to some embodiments of the present disclosure, a method ofconfiguring robot fleets with additive manufacturing capabilities isdisclosed. The method includes receiving a request for a robotic fleetto perform a job. The method further includes determining a jobdefinition data structure based on the request, the job definition datastructure defining a set of tasks that are to be performed inperformance of the job. The method further includes determining arobotic fleet configuration data structure corresponding to the jobbased on the set of tasks and a fleet resource inventory that indicatesa plurality of additive manufacturing systems that can be provisionedwith a set of fleet resources, and for each respective additivemanufacturing system, a set of 3D printing requirements, printinginstructions that define configuring an on-demand production system for3D printing, and a status of the additive manufacturing system, whereinthe robotic fleet configuration data structure assigns one or moreadditive manufacturing systems selected from the fleet resourceinventory to one or more of the set of tasks defined in the jobdefinition data structure. The method further includes determining arespective provisioning configuration for each respective additivemanufacturing system based on the respective task to which the additivemanufacturing system is assigned, the set of 3D printing requirements,the printing instructions, and the respective status of the additivemanufacturing system. The method further includes provisioning therespective additive manufacturing system based on the respectiveprovisioning configuration and a set of additive manufacturing systemprovisioning rules that are accessible to an intelligence layer toensure that provisioned additive manufacturing systems comply with theprovisioning rules. The method further includes deploying the roboticfleet based on the robotic fleet configuration data structure to performthe job.

In some embodiments, provisioning the respective additive manufacturingsystem includes provisioning a 3D printing capable robot. In someembodiments, the respective provisioning configuration for eachrespective additive manufacturing system includes a set of 3D printinginstructions for at least one of a job-specific end effector or anadaptor based on a context of the task to which the additivemanufacturing system is assigned. In some embodiments, the robotic fleetconfiguration data structure assigns control of at least onetransportable 3D printing additive manufacturing system to at least onerobot operating unit.

In some embodiments, determining the robotic fleet configuration datastructure is further based on availability and job site locality of 3Dprinting resources. In some of these embodiments, at least one of theavailability or job site locality of the 3D printing resource isidentified by a logistics system of the platform. In some embodiments,determining the robotic fleet configuration data structure includesassignment of at least one additive manufacturing system indicated inthe fleet resource inventory based on proximity to a job site for therequested job.

In some embodiments, determining a respective provisioning configurationfor each respective additive manufacturing system includes use of anartificial intelligence system to automate design for 3D printing of oneor more robotic accessories. In some of these embodiments, theartificial intelligence system automates design for 3D printing based oncontextual task recognition. Additionally or alternatively, theartificial intelligence system automates design for 3D printing based onautomated shape recognition capabilities. Additionally or alternatively,provisioning the respective additive manufacturing system includesprovisioning a 3D printing control capability to produce an end effectorbased on a visual and sensed analysis of an object for manipulation ofwhich the end effector is to be 3D printed.

In some embodiments, deploying the robotic fleet includes use of a fleetconfiguration scheduling resource of the platform for allocation of therespective additive manufacturing system to perform the job. In someembodiments, deploying the robotic fleet includes deploying a 3Dprinting robot to a smart container for remote, on-demand additivemanufacturing. In some embodiments, determining a respectiveprovisioning configuration for each respective additive manufacturingsystem is further based on one or more keywords of the job definitiondata structure that are indicative of an operating condition for therespective additive manufacturing system. In some embodiments, deployingthe robotic fleet includes deploying a set of autonomous 3D printingadditive manufacturing system to points of service work indicated in thejob definition data structure. In some embodiments, determining arespective provisioning configuration for each respective additivemanufacturing system includes configuring a 3D printing system toreceive a tokenized instance of a set of 3D printing instructionsassociated with a corresponding token on a distributed ledger. In someembodiments, deploying the robotic fleet includes deploying therespective additive manufacturing system as a 3D printing resourceshared among a plurality of tasks.

In some embodiments, provisioning the respective additive manufacturingsystem includes interacting with at least one of a fleet operatingsystem, a fleet configuration system, a fleet resource schedulingsystem, and a fleet utilization system. In some of these embodiments,interacting includes ensuring that the provisioning rules are followed.In some embodiments, the provisioning rules are defined in a governancestandards library and an intelligence service ensures that theprovisioned resources comply with the provisioning rules.

According to some embodiments of the present disclosure, a dynamicvision system for robot fleet management is disclosed. The systemincludes an optical assembly including a lens containing a liquid,wherein the lens is deformable to generate variable focus for the lens,and wherein the optical assembly is configured to capture optical data.The system further includes a robot fleet management platform having acontrol system configured to adjust one or more optical parameters,wherein the one or more optical parameters modify the variable focus ofthe lens while the optical assembly captures current optical datarelating to a robotic fleet. The system further includes a processingsystem configured to train a machine learning model to recognize anobject relating to the robotic fleet using training data generated fromthe optical data captured by the optical assembly, wherein the opticaldata includes the current optical data relating to the robotic fleet.

In some embodiments, the optical data captured by the optical assemblyincludes optical data that is out-of-focus with respect to an objectbeing optically captured by the optical assembly. In some embodiments,the recognition of an object relating to the robotic fleet is comparedto a stored fleet resource configuration comprised of a plurality ofobjects. In some of these embodiments, the comparison of the recognizedobject to the stored fleet resource configuration is quantified as anumeric score, wherein the numeric score represents the degree of matchbetween the recognized object and that object type's position in thestored fleet resource configuration. In some of these embodiments, thenumeric score is compared against a stored numeric score threshold,wherein the numeric score threshold represents a minimum degree of matchbetween the recognized object and that object type's position in thestored fleet resource configuration. In some of these embodiments, therobotic fleet management platform generates an alert upon detection ofthe numeric score not meeting or exceeding the stored numeric scorethreshold.

In some embodiments, the robotic fleet management platform pausesrobotic activity of at least one robotic apparatus upon detection of thenumeric score not meeting or exceeding the stored numeric scorethreshold. In some embodiments, the optical parameters deform the lensfrom an original state by applying an electrical current to the lens. Insome embodiments, the optical parameters adjust the variable focus ofthe lens at a predetermined frequency. In some embodiments, the opticalparameters adjust the variable focus of the lens from a first focalstate to a second focal state different than the first focal state,wherein the training data includes optical data captured in the firstfocal state, and wherein the training data incorporates feedback datasuch that the training data includes optical data captured in the firstfocal state and the second focal state.

According to some embodiments of the present disclosure, an informationtechnology system for a distributed manufacturing network is disclosed.The system includes an additive manufacturing management platformconfigured to manage process workflows for a set of distributedmanufacturing network entities associated with the distributedmanufacturing network, wherein one of the process workflows includes adesign stage, a modeling stage, a printing stage, and a supply chainstage, wherein the modeling stage includes a digital twin modelingsystem defined at least in part by at least one of a product instructionor the control tower instruction to encode a set of digital twinsrepresenting a product for use by the additive manufacturing managementplatform. The system further includes an artificial intelligence systemexecutable by a data processing system in communication with theadditive manufacturing management platform, wherein the artificialintelligence system is trained to generate process parameters for theprocess workflows managed by the additive manufacturing managementplatform using data collected from the distributed manufacturing networkentities. The system further includes a control system configured toadjust the process parameters during an additive manufacturing processperformed by at least one of the distributed manufacturing networkentities.

In some embodiments, the set of distributed manufacturing networkentities includes: a first additive manufacturing unit configured toperform a first additive manufacturing process; and a second additivemanufacturing unit configured to perform a second additive manufacturingprocess, wherein the first additive manufacturing process is differentthan the second additive manufacturing process.

In some embodiments, the training data includes: (i) outcomes; (ii) datacollected; and (iii) prior/historical process parameters. In someembodiments, the additive manufacturing process is a hybrid taskrequiring at least two different types of additive manufacturing units.In some embodiments, the additive manufacturing management platform iscloud-based. In some embodiments, the artificial intelligence system isdistributed across more than one distributed manufacturing networkentity. In some embodiments, the digital twins representing a productare used by the additive manufacturing management platform tomanufacture a physical replica of the digitally represented product. Insome embodiments, the artificial intelligence system includes anadaptive intelligence system in communication with a plurality ofsensors and configured to receive current sensor data from the pluralityof sensors for use in encoding the set of digital twins. In someembodiments, the artificial intelligence system is distributed acrossmore than one distributed manufacturing network entities from the set ofdistributed manufacturing network entities. In some embodiments, therepresentation of the product is a simulated future condition state ofthe product.

An autonomous futures contract orchestration platform includes a set ofone or more processors programmed with a set of non-transitorycomputer-readable instructions to collectively execute receiving, from adata source, an indication associated with a product that relates to anentity that at least one of purchases or sells the product. They furtherexecute predicting a baseline cost of at least one of purchasing orselling the product at a future point in time based on the indication.They further execute retrieving a futures cost, at a current point intime, of a futures contract for an obligation to the at least one ofpurchasing or selling the product for at least one of delivery orperformance of the product at the future point in time. They furtherexecute executing a smart contract for the futures contract based on thebaseline cost and the futures cost. They further execute orchestratingthe at least one of delivery or performance of the product at the futurepoint in time.

In other features, the autonomous futures contract orchestrationplatform includes a risk data structure indicating an amount of risk theentity is willing to accept with respect to the baseline cost and thefutures cost. The computer-readable instructions collectively executeexecuting the smart contract based on the risk data structure to atleast one of manage or mitigate risk. In other features, the autonomousfutures contract orchestration platform includes a robotic processautomation system for demand-side planning to orchestrate the smartfutures contract. In other features, the autonomous futures contractorchestration platform includes a robotic agent configured to deriskwith respect to the futures contract and the smart contract. In otherfeatures, the autonomous futures contract orchestration platformincludes a system for performing circular economy optimization based onfutures pricing of goods. In other features, the computer-readableinstructions collectively execute initializing a robotic processautomation system trained to execute the smart contract and executingthe smart contract using the robotic process automation system. In otherfeatures, the indication is of at least one of an event occurrence, aphysical condition of an item, or a potential demand increase.

An autonomous futures contract orchestration platform includes a set ofone or more processors programmed with a set of non-transitorycomputer-readable instructions to collectively execute retrieving afutures cost, at a current point in time, of a futures contract for anobligation to at least one of purchase or sell a product for at leastone of delivery or performance of the product to an entity at a futurepoint in time. They further execute predicting a baseline cost to theentity of the at least one of purchasing or selling the product at thefuture point in time. They further execute executing a smart contractfor the futures contract based on the baseline cost and the futurescost. They further execute orchestrating the at least one of delivery orperformance of the product to the entity at the future point in time.

A computerized method for autonomous future contract orchestrationincludes receiving, from a data source, an indication associated with aproduct that relates to an entity that at least one of purchases orsells the product. The method includes predicting a baseline cost of atleast one of purchasing or selling the product at a future point in timebased on the indication. The method includes retrieving a futures cost,at a current point in time, of a futures contract for an obligation tothe at least one of purchasing or selling the product for at least oneof delivery or performance of the product at the future point in time.The method includes executing a smart contract for the futures contractbased on the baseline cost and the futures cost. The method includesorchestrating the at least one of delivery or performance of the productat the future point in time.

In other features, the computerized method includes retrieving a riskdata structure indicating an amount of risk the entity is willing toaccept with respect to the baseline cost and the futures cost andexecuting the smart contract based on the risk data structure to atleast one of manage or mitigate risk. In other features, thecomputerized method includes demand-side planning using a roboticprocess automation system and orchestrating the smart futures contractbased on the demand-side planning. In other features, the computerizedmethod includes derisking with respect to the futures contract and thesmart contract using a robotic agent. In other features, thecomputerized method includes executing a system for performing circulareconomy optimization based on futures pricing of goods. In otherfeatures, the computerized method includes initializing a roboticprocess automation system trained to execute the smart contract andexecuting the smart contract using the robotic process automationsystem. In other features, retrieving the indication includes retrievingat least one of an event occurrence, a physical condition of an item, ora potential demand increase.

An autonomous futures contract orchestration platform includes a set ofone or more processors programmed with a set of non-transitorycomputer-readable instructions to collectively execute receiving, from adata source, an indication associated with a product that relates to anentity that at least one of purchases or sells the product. They furtherexecute predicting a baseline cost of at least one of purchasing orselling the product at a future point in time based on the indication.They further execute retrieving a futures cost, at a current point intime, of a futures contract for the product. They further executegenerating a risk threshold based on a predefined risk tolerance of theentity indicating a difference between the baseline cost and the futurescost. They further execute executing a smart contract for the futurescontract based on the baseline cost, the futures cost, and the riskthreshold.

In other features, the set of one or more processors are furtherprogrammed to collectively execute generating the risk threshold basedon at least one of hedging for or providing improved outcomes afteradverse contingencies. In other features, the set of one or moreprocessors are further programmed to collectively execute generating therisk threshold based on at least one of: shortages in supply, supplychain disruptions, changes in demand, changes in prices of inputs, orchanges in market prices as the adverse contingencies. In otherfeatures, the set of one or more processors are further programmed tocollectively execute predicting the baseline cost based on providingoperational efficiencies. In other features, the set of one or moreprocessors are further programmed to collectively execute predicting thebaseline cost based on at least one of insuring availability of itemsbased on plans or insuring availability of items based on availabilitypredictions as the operational efficiencies.

In other features, the set of one or more processors are furtherprogrammed to collectively execute executing the smart contract based onimproving returns. In other features, the set of one or more processorsare further programmed to collectively execute executing the smartcontract based on obtaining inputs at more favorable prices than thebaseline cost indicates. In other features, the set of one or moreprocessors are further programmed to collectively execute executing thesmart contract that interacts with futures markets associated with thefutures contract. In other features, the set of one or more processorsare further programmed to collectively execute executing the smartcontract to engage with at least one of futures or options involving atleast one of commodities, equities, currencies, or energy associatedwith the futures contract.

A computerized method for autonomous futures contract orchestrationincludes receiving, from a data source, an indication associated with aset of items that are provided at least one of by or within a valuechain network. The method includes predicting a baseline cost associatedwith the set of items at a future point in time based on the indication.The method includes retrieving a futures cost, at a current point intime, of a futures contract associated with the set of items. The methodincludes generating a risk threshold based on a predefined risktolerance of an entity of the value chain network, the risk thresholdindicating a difference between the baseline cost and the futures cost.The method includes executing a smart contract for the futures contractbased on the baseline cost, the futures cost, and the risk threshold.

In other features, generating the risk threshold includes generating therisk threshold based on at least one of hedging for or providingimproved outcomes after adverse contingencies. In other features,generating the risk threshold includes generating the risk thresholdbased on at least one of: shortages in supply, supply chain disruptions,changes in demand, changes in prices of inputs, or changes in marketprices as the adverse contingencies. Predicting the baseline costincludes predicting the baseline cost based on providing operationalefficiencies. In other features, predicting the baseline cost includespredicting the baseline cost based on at least one of insuringavailability of items based on plans or insuring availability of itemsbased on availability predictions as the operational efficiencies.

In other features, executing the smart contract includes executing thesmart contract based on improving returns. In other features, executingthe smart contract includes executing the smart contract based onobtaining inputs at more favorable prices than the baseline costindicates. In other features, executing the smart contract includesexecuting a smart contract that interacts with futures marketsassociated with the futures contract. In other features, executing thesmart contract includes executing the smart contract to engage with atleast one of futures or options involving at least one of commodities,equities, currencies, or energy associated with the futures contract.

A system for managing future costs associated with a product includes afuture requirement system programmed to estimate an amount of resourcesrequired for manufacturing, distributing, and selling the product at afuture point in time. The system includes an adverse contingency systemconfigured to identify adverse contingencies and calculate changes incosts associated with obtaining the amount of resources at the futurepoint in time. The system includes a smart contract system programmed toautonomously configure and execute a smart futures contract based on theamount of resources required and on the changes in costs to manage thefuture costs associated with the product.

In other features, the smart contract system is further programmed toexecute the smart futures contract based on at least one of hedging foror providing improved outcomes after the adverse contingencies. In otherfeatures, the adverse contingency system is further configured toestimate probabilities of at least one of: shortages in supply, supplychain disruptions, changes in demand, changes in prices of inputs, orchanges in market prices as the adverse contingencies.

In other features, the adverse contingency system is further configuredto estimate probabilities of at least one of: macro-economic factors,geopolitical disruptions, disruptions due to weather or climate,epidemics, pandemics, or counterparty risks as the adversecontingencies. In other features, the smart contract system isprogrammed with a robotic agent that configures terms and conditions forthe smart futures contract. In other features, the smart contract systemis programmed to set prices, delivery times, and delivery locationsrequired in order to provide a pre-determined inventory of an item inresponse to the adverse contingencies. In other features, the smartcontract system is programmed to configure at least one of parts,components, fuel, or materials required to provide a pre-determinedinventory of an item as a set of inputs with the robotic agent. In otherfeatures, the smart contract system is programmed to train the roboticagent on a training set of interactions of a set of expert procurementprofessionals with a set of inputs.

In other features, the smart contract system is programmed to train therobotic agent with at least one of demand forecasts, inventoryforecasts, demand elasticity curves, predictions of competitivebehavior, supply chain predictions as demand planning inputs of the setof inputs. In other features, the smart contract system is programmed totrain the robotic agent with interactions within an enterprise demandplanning software suite as the set of inputs. In other features, thesmart contract system is programmed to train the robotic agent tointeract with a set of demand models that at least one of forecastdemand factors, forecast supply factors, forecast pricing factors,forecast anticipated equilibria between supply and demand, generateestimates of appropriate inventory, generate recommendations for supply,or generate recommendations for distribution. In other features, thesmart contract system is further programmed to configure the smartcontract to automatically execute to obtain commitments for supply inresponse to discovery of a pre-defined market condition associated withthe adverse contingency.

A computerized method for managing future costs associated with aproduct includes estimating an amount of resources required formanufacturing, distributing, and selling the product at a future pointin time. The method includes identifying adverse contingencies. Themethod includes calculating changes in costs associated with obtainingthe amount of resources at the future point in time. The method includesautonomously configuring and executing a smart futures contract based onthe amount of resources required and on the changes in costs to managethe future costs associated with the product.

In other features, executing the smart contract includes executing thesmart futures contract based on at least one of hedging for or providingimproved outcomes after the adverse contingencies. In other features,the computerized method includes estimating probabilities of at leastone of: shortages in supply, supply chain disruptions, changes indemand, changes in prices of inputs, or changes in market prices as theadverse contingencies. In other features, the computerized methodincludes estimating probabilities of at least one of: macro-economicfactors, geopolitical disruptions, disruptions due to weather orclimate, epidemics, pandemics, or counterparty risks as the adversecontingencies.

In other features, the computerized method includes configuring termsand conditions for the smart futures contract with a robotic agent. Inother features, the computerized method includes configuring at leastone of parts, components, fuel, or materials required to provide apre-determined inventory of an item as a set of inputs with the roboticagent. In other features, the computerized method includes training therobotic agent on a training set of interactions of a set of expertprocurement professionals with a set of inputs. In other features, thecomputerized method includes training the robotic agent to interact witha set of demand models that at least one of forecast demand factors,forecast supply factors, forecast pricing factors, forecast anticipatedequilibria between supply and demand, generate estimates of appropriateinventory, generate recommendations for supply, or generaterecommendations for distribution.

A raw material system includes a product manufacturing demand estimationsystem programmed to calculate an expected demand for a product at afuture point in time. The system includes an environment detectionsystem configured to identify at least one of an environmental conditionor an environmental event. The system includes a raw material productionsystem programmed to estimate a raw material availability at the futurepoint in time based on the expected demand and the at least one of theenvironmental condition or the environmental event. The system includesa raw material requirement system programmed to calculate a required rawmaterial amount to manufacture the product at the future point in timebased on the expected demand and on the at least one of theenvironmental condition or the environmental event. The system includesa raw material procurement system programmed to autonomously configure afutures contract for procurement of at least a portion of the requiredraw material amount in response to the required raw material amountcalculation exceeding the raw material availability estimation.

In other features, the raw material production system is furtherprogrammed to estimate a probability that the raw material availabilitywill decrease based on a rise in demand outpacing a production increase.In other features, the raw material requirement system is furtherprogrammed with a demand aggregation service configured to monitor ademand response across a plurality of systems. In other features, thedemand aggregation service is further configured to monitor the demandresponse as changes in at least one of supply, price changes,customization, pricing, or advertising. In other features, the rawmaterial system includes a risk tolerance system configured to retrievea pre-determined risk tolerance of an entity that procures the rawmaterial. The raw material procurement system is further programmed toautonomously configure the futures contract based at least in part onthe pre-determined risk tolerance. In other features, the raw materialprocurement system is further configured to execute a smart contract forthe futures contract. In other features, the raw material systemincludes a digital wallet coupled with the raw material procurementsystem to enable payments associated with the smart contract. In otherfeatures, the raw material procurement system is further configured witha robotic process automation (RPA) service to facilitate automation ofproducing and validating the smart contract. In other features, the RPAservice is configured to automate processes based on observations ofhuman interactions with hardware elements and with software elements.

In other features, the raw material procurement system is furtherconfigured to configure the smart contract to interact with adistribution system to secure at least one of delivery, storage, orhandling of the raw materials through the distribution system. In otherfeatures, the raw material procurement system is further configured toconfigured the smart contract to interact with a logistics reservationsfutures system to secure future logistics services. In other features,the raw material procurement system is further configured to configurethe smart contract to secure at least one of port docking reservations,shipping container reservations, trucking reservations, warehouse spacerental, or canal passage rental as the future logistics services. Inother features, the raw materials include at least one of copper, steel,iron, or lithium.

A computerized method for raw material procurement includes calculatingan expected demand for a product at a future point in time. The methodincludes identifying at least one of an environmental condition or anenvironmental event. The method includes estimating a raw materialavailability of a raw material at the future point in time based on theexpected demand and the at least one of the environmental condition orthe environmental event. The method includes calculating a required rawmaterial amount of the raw material to manufacture the product at thefuture point in time based on the expected demand and on the at leastone of the environmental condition or the environmental event. Themethod includes autonomously configuring a futures contract forprocurement of at least a portion of the required raw material amount inresponse to the required raw material amount calculation exceeding theraw material availability estimation.

In other features, the computerized method includes estimating aprobability that the raw material availability will decrease based on arise in demand outpacing a production increase. In other features, thecomputerized method includes monitoring a demand response across aplurality of systems. In other features, monitoring the demand responsefurther includes to monitoring the demand response as changes in atleast one of supply, price changes, customization, pricing, oradvertising. In other features, the computerized method includesretrieving a pre-determined risk tolerance of an entity that procuresthe raw material. Autonomously configuring the futures contract is basedat least in part on the pre-determined risk tolerance. In otherfeatures, the computerized method includes executing a smart contractfor the futures contract. In other features, the computerized methodincludes engaging a digital wallet to enable payments associated withthe smart contract.

A system for product replacement includes a product logistics system fora product in a product condition. The system includes an exposure datacollection system configured to collect exposure data indicating atleast one of an event or an environmental condition that may impact theproduct condition of the product. The system includes a replacementdetermination system programmed to calculate a probability for the needto replace the product based on the at least one of the event or theenvironmental condition. The system includes a replacement procurementsystem programmed to autonomously configure an option-type futurescontract for replacement of the product based on the probability for theneed to replace the product.

In other features, the system includes a smart contract systemprogrammed to autonomously configure a smart contract to securereplacement of the product based on the option-type futures contract. Inother features, the smart contract system configures the smart contractto have a duration of option based on estimating a time until an actualdetermination of the need to replace the product based on physicalexamination may be performed. In other features, the smart contractsystem configures the smart contract to have the duration of optionsfurther based on a probability of catastrophic loss indicated by theprobability for the need to replace the product. In other features, thesystem includes a replacement alternatives system programmed toconfigure an alternative smart contract that offers alternatives toreplacement of the product to at least one of a purchaser of, an ownerof, or an insurer with a security interest in the product. In otherfeatures, the replacement alternatives system is programmed to configurethe alternative smart contract that offers a refund of a purchase priceof the product. In other features, the replacement alternatives systemis programmed to configure the alternative smart contract that offersalternative goods or services. In other features, the replacementalternatives system is programmed to configure the alternative smartcontract that offers incentives to accept a delayed delivery of theproduct.

In other features, the system includes a future price renegotiationsystem programmed to renegotiate a set of future prices based on acurrent market state and on the exposure data. In other features, thefuture price renegotiation system is further programmed to renegotiatethe set of future prices in response to the exposure data indicating alikelihood of widespread supply chain disruptions for goods or servicesassociated with the product. In other features, the system includes anartificial intelligence (AI) system trained on historical data sets topredict the probability that the product will need to be replaced basedon the exposure data. In other features, the AI system is trained topredict the impact of the need for replacement. In other features, theAI system is trained to predict the impact of the need based on at leastone of an impact of delays or reduced supply on pricing. In otherfeatures, the exposure data collection system is further configured tocollect the exposure data from sensors disposed on at least one of theproduct, a package for the product, a transport vehicle in which theproduct is located, or proximal infrastructure.

A computerized method for product replacement of a product in a productcondition includes collecting exposure data indicating at least one ofan event or an environmental condition that may impact the productcondition of the product. The method includes calculating a probabilityfor the need to replace the product based on the at least one of theevent or the environmental condition. The method includes autonomouslyconfiguring an option-type futures contract for replacement of theproduct based on the probability for the need to replace the product.

In other features, the computerized method includes autonomouslyconfiguring a smart contract to secure replacement of the product basedon the option-type futures contract. In other features, the computerizedmethod includes estimating a time until an actual determination of theneed to replace the product will be performed. Configuring the smartcontract includes configuring the smart contract to have a duration ofoption based on the time until the actual determination will beperformed. In other features, configuring the smart contract includesconfiguring the smart contract to have the duration of options furtherbased on a probability of catastrophic loss indicated by the probabilityfor the need to replace the product. In other features, the computerizedmethod includes configuring an alternative smart contract that offersalternatives to replacement of the product to at least one of apurchaser of, an owner of, or an insurer with a security interest in theproduct. In other features, configuring the alternative smart contractincludes configuring the alternative smart contract that offers a refundof a purchase price of the product.

A more complete understanding of the disclosure will be appreciated fromthe description and accompanying drawings and the claims, which follow.All documents referenced herein are hereby incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a betterunderstanding of the disclosure, illustrate embodiments of thedisclosure and together with the description serve to explain the manyaspects of the disclosure. In the drawings:

FIG. 1 is a block diagram showing prior art relationships of variousentities and facilities in a supply chain.

FIG. 2 is a block diagram showing components and interrelationships ofsystems and processes of a value chain network in accordance with thepresent disclosure.

FIG. 3 is another block diagram showing components andinterrelationships of systems and processes of a value chain network inaccordance with the present disclosure.

FIG. 4 is a block diagram showing components and interrelationships ofsystems and processes of a digital products network of FIGS. 2 and 3 inaccordance with the present disclosure.

FIG. 5 is a block diagram showing components and interrelationships ofsystems and processes of a value chain network technology stack inaccordance with the present disclosure.

FIG. 6 is a block diagram showing a platform and relationships fororchestrating controls of various entities in a value chain network inaccordance with the present disclosure.

FIG. 7 is a block diagram showing components and relationships inembodiments of a value chain network management platform in accordancewith the present disclosure.

FIG. 8 is a block diagram showing components and relationships of valuechain entities managed by embodiments of a value chain networkmanagement platform in accordance with the present disclosure.

FIG. 9 is a block diagram showing network relationships of entities in avalue chain network in accordance with the present disclosure.

FIG. 10 is a block diagram showing a set of applications supported byunified data handling layers in a value chain network managementplatform in accordance with the present disclosure.

FIG. 11 is a block diagram showing components and relationships inembodiments of a value chain network management platform in accordancewith the present disclosure.

FIG. 12 is a block diagram showing components and relationships of adata storage layer in embodiments of a value chain network managementplatform in accordance with the present disclosure.

FIG. 13 is a block diagram showing components and relationships of anadaptive intelligent systems layer in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 14 is a block diagram that depicts providing adaptive intelligencesystems for coordinated intelligence for sets of demand and supplyapplications for a category of goods in accordance with the presentdisclosure.

FIG. 15 is a block diagram that depicts providing hybrid adaptiveintelligence systems for coordinated intelligence for sets of demand andsupply applications or a category of goods in accordance with thepresent disclosure.

FIG. 16 is a block diagram that depicts providing adaptive intelligencesystems for predictive intelligence for sets of demand and supplyapplications for a category of goods in accordance with the presentdisclosure.

FIG. 17 is a block diagram that depicts providing adaptive intelligencesystems for classification intelligence for sets of demand and supplyapplications for a category of goods in accordance with the presentdisclosure.

FIG. 18 is a block diagram that depicts providing adaptive intelligencesystems to produce automated control signals for sets of demand andsupply applications for a category of goods in accordance with thepresent disclosure.

FIG. 19 is a block diagram that depicts training artificialintelligence/machine learning systems to produce information routingrecommendations for a selected value chain network in accordance withthe present disclosure.

FIG. 20 is a block diagram that depicts a semi-sentient problemrecognition system for recognition of pain points/problem states in avalue chain network in accordance with the present disclosure.

FIG. 21 is a block diagram that depicts a set of artificial intelligencesystems operating on value chain information to enable automatedcoordination of value chain activities for an enterprise in accordancewith the present disclosure.

FIG. 22 is a block diagram showing components and relationships involvedin integrating a set of digital twins in an embodiment of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 23 is a block diagram showing a set of digital twins involved inembodiments of a value chain network management platform in accordancewith the present disclosure.

FIG. 24 is a block diagram showing components and relationships ofentity discovery and management systems in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 25 is a block diagram showing components and relationships of arobotic process automation system in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 26 is a block diagram showing components and relationships of a setof opportunity miners in an embodiment of a value chain networkmanagement platform in accordance with the present disclosure.

FIG. 27 is a block diagram showing components and relationships of a setof edge intelligence systems in embodiments of a value chain networkmanagement platform in accordance with the present disclosure.

FIG. 28 is a block diagram showing components and relationships in anembodiment of a value chain network management platform in accordancewith the present disclosure.

FIG. 29 is a block diagram showing additional details of components andrelationships in embodiments of a value chain network managementplatform in accordance with the present disclosure.

FIG. 30 is a block diagram showing components and relationships in anembodiment of a value chain network management platform that enablescentralized orchestration of value chain network entities in accordancewith the present disclosure.

FIG. 31 is a block diagram showing components and relationships of aunified database in an embodiment of a value chain network managementplatform in accordance with the present disclosure.

FIG. 32 is a block diagram showing components and relationships of a setof unified data collection systems in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 33 is a block diagram showing components and relationships of a setof Internet of Things monitoring systems in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 34 is a block diagram showing components and relationships of amachine vision system and a digital twin in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 35 is a block diagram showing components and relationships of a setof adaptive edge intelligence systems in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 36 is a block diagram showing additional details of components andrelationships of a set of adaptive edge intelligence systems inembodiments of a value chain network management platform in accordancewith the present disclosure.

FIG. 37 is a block diagram showing components and relationships of a setof unified adaptive intelligence systems in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 38 is a schematic of a system configured to train an artificialsystem that is leveraged by a value chain system using real worldoutcome data and a digital twin system according to some embodiments ofthe present disclosure.

FIG. 39 is a schematic of a system configured to train an artificialsystem that is leveraged by a container fleet management system usingreal world outcome data and a digital twin system according to someembodiments of the present disclosure.

FIG. 40 is a schematic of a system configured to train an artificialsystem that is leveraged by a logistics design system using real worldoutcome data and a digital twin system according to some embodiments ofthe present disclosure.

FIG. 41 is a schematic of a system configured to train an artificialsystem that is leveraged by a packaging design system using real worldoutcome data and a digital twin system according to some embodiments ofthe present disclosure.

FIG. 42 is a schematic of a system configured to train an artificialsystem that is leveraged by a waste mitigation system using real worldoutcome data and a digital twin system according to some embodiments ofthe present disclosure.

FIG. 43 is a schematic illustrating an example of a portion of aninformation technology system for value chain artificial intelligenceleveraging digital twins according to some embodiments of the presentdisclosure.

FIG. 44 is a block diagram showing components and relationships of a setof intelligent project management facilities in embodiments of a valuechain network management platform in accordance with the presentdisclosure.

FIG. 45 is a block diagram showing components and relationships of anintelligent task recommendation system in embodiments of a value chainnetwork management platform in accordance with the present disclosure.

FIG. 46 is a block diagram showing components and relationships of arouting system among nodes of a value chain network in embodiments of avalue chain network management platform in accordance with the presentdisclosure.

FIG. 47 is a block diagram showing components and relationships of adashboard for managing a set of digital twins in embodiments of a valuechain network management platform.

FIG. 48 is a block diagram showing components and relationships inembodiments of a value chain network management platform that uses amicroservices architecture.

FIG. 49 is a block diagram showing components and relationships of anInternet of Things data collection architecture and sensorrecommendation system in embodiments of a value chain network managementplatform.

FIG. 50 is a block diagram showing components and relationships of asocial data collection architecture in embodiments of a value chainnetwork management platform.

FIG. 51 is a block diagram showing components and relationships of acrowdsourcing data collection architecture in embodiments of a valuechain network management platform.

FIG. 52 is a diagrammatic view that depicts embodiments of a set ofvalue chain network digital twins representing virtual models of a setof value chain network entities in accordance with the presentdisclosure.

FIG. 53 is a diagrammatic view that depicts embodiments of a warehousedigital twin kit system in accordance with the present disclosure.

FIG. 54 is a diagrammatic view that depicts embodiments of a stress testperformed on a value chain network in accordance with the presentdisclosure.

FIG. 55 is a diagrammatic view that depicts embodiments of methods usedby a machine for detecting faults and predicting any future failures ofthe machine in accordance with the present disclosure.

FIG. 56 is a diagrammatic view that depicts embodiments of deployment ofmachine twins to perform predictive maintenance on a set of machines inaccordance with the present disclosure.

FIG. 57 is a schematic illustrating an example of a portion of a systemfor value chain customer digital twins and customer profile digitaltwins according to some embodiments of the present disclosure.

FIG. 58 is a schematic illustrating an example of an advertisingapplication that interfaces with the adaptive intelligent systems layerin accordance with the present disclosure.

FIG. 59 is a schematic illustrating an example of an e-commerceapplication integrated with the adaptive intelligent systems layer inaccordance with the present disclosure.

FIG. 60 is a schematic illustrating an example of a demand managementapplication integrated with the adaptive intelligent systems layer inaccordance with the present disclosure.

FIG. 61 is a schematic illustrating an example of a portion of a systemfor value chain smart supply component digital twins according to someembodiments of the present disclosure.

FIG. 62 is a schematic illustrating an example of a risk managementapplication that interfaces with the adaptive intelligent systems layerin accordance with the present disclosure.

FIG. 63 is a diagrammatic view of maritime assets associated with avalue chain network management platform including components of a portinfrastructure in accordance with the present disclosure.

FIGS. 64 and 65 are diagrammatic views of maritime assets associatedwith a value chain network management platform including components of aship in accordance with the present disclosure.

FIG. 66 is a diagrammatic view of maritime assets associated with avalue chain network management platform including components of a bargein accordance with the present disclosure.

FIG. 67 is a diagrammatic view of maritime assets associated with avalue chain network management platform including those involved inmaritime events, legal proceedings and making use of geofencedparameters in accordance with the present disclosure.

FIG. 68 is a schematic illustrating an example environment of theenterprise and executive control tower and management platform,including data sources in communication therewith, according to someembodiments of the present disclosure.

FIG. 69 is a schematic illustrating an example set of components of theenterprise control tower and management platform according to someembodiments of the present disclosure.

FIG. 70 is a schematic illustrating and example of an enterprise datamodel according to some embodiments of the disclosure.

FIG. 71 is a schematic illustrating examples of different types ofenterprise digital twins, including executive digital twins, in relationto the data layer, processing layer, and application layer of theenterprise digital twin framework according to some embodiments of thepresent disclosure.

FIG. 72 is a schematic illustrating an example implementation of theenterprise and executive control tower and management platform accordingto some embodiments of the present disclosure.

FIG. 73 is a flow chart illustrating an example set of operations forconfiguring and serving an enterprise digital twin.

FIG. 74 illustrates an example set of operations of a method forconfiguring an organizational digital twin.

FIG. 75 illustrates an example set of operations of a method forgenerating an executive digital twin.

FIGS. 76-103 are schematic diagrams of embodiments of neural net systemsthat may connect to, be integrated in, and be accessible by the platformfor enabling intelligent transactions including ones involving expertsystems, self-organization, machine learning, artificial intelligenceand including neural net systems trained for pattern recognition, forclassification of one or more parameters, characteristics, or phenomena,for support of autonomous control, and other purposes in accordance withembodiments of the present disclosure.

FIG. 104 is a schematic illustrating an example intelligence servicessystem according to some embodiments of the present disclosure.

FIG. 105 is a schematic illustrating an example neural network withmultiple layers according to some embodiments of the present disclosure.

FIG. 106 is a schematic illustrating an example convolutional neuralnetwork (CNN) according to some embodiments of the present disclosure.

FIG. 107 is a schematic illustrating an example neural network forimplementing natural language processing according to some embodimentsof the present disclosure.

FIG. 108 is a schematic illustrating an example reinforcementlearning-based approach for executing one or more tasks by a mobilesystem according to some embodiments of the present disclosure.

FIG. 109 is a schematic illustrating an example physical orientationdetermination chip according to some embodiments of the presentdisclosure.

FIG. 110 is a schematic illustrating an example network enhancement chipaccording to some embodiments of the present disclosure.

FIG. 111 is a schematic illustrating an example diagnostic chipaccording to some embodiments of the present disclosure.

FIG. 112 is a schematic illustrating an example governance chipaccording to some embodiments of the present disclosure.

FIG. 113 is a schematic illustrating an example prediction,classification, and recommendation chip according to some embodiments ofthe present disclosure.

FIG. 114 is a diagrammatic view illustrating an example environment ofan autonomous additive manufacturing platform according to someembodiments of the present disclosure.

FIG. 115 is a schematic illustrating an example implementation of anautonomous additive manufacturing platform for automating and optimizingthe digital production workflow for metal additive manufacturingaccording to some embodiments of the present disclosure.

FIG. 116 is a flow diagram illustrating the optimization of differentparameters of an additive manufacture process according to someembodiments of the present disclosure.

FIG. 117 is a schematic view illustrating a system for learning on datafrom an autonomous additive manufacturing platform to train anartificial learning system to use digital twins for classification,predictions and decision making according to some embodiments of thepresent disclosure.

FIG. 118 is a schematic illustrating an example implementation of anautonomous additive manufacturing platform including various componentsalong with other entities of a distributed manufacturing networkaccording to some embodiments of the present disclosure.

FIG. 119 is a schematic illustrating an example implementation of anautonomous additive manufacturing platform for automating and managingmanufacturing functions and sub-processes including process and materialselection, hybrid part workflows, feedstock formulation, part designoptimization, risk prediction and management, marketing and customerservice according to some embodiments of the present disclosure.

FIG. 120 is a diagrammatic view of a distributed manufacturing networkenabled by an autonomous additive manufacturing platform and built on adistributed ledger system according to some embodiments of the presentdisclosure.

FIG. 121 is a schematic illustrating an example implementation of adistributed manufacturing network where the digital thread data istokenized and stored in a distributed ledger so as to ensuretraceability of parts printed at one or more manufacturing nodes in thedistributed manufacturing network according to some embodiments of thepresent disclosure.

FIG. 122 is a diagrammatic view illustrating an example implementationof a conventional computer vision system for creating an image of anobject of interest.

FIG. 123 is a schematic illustrating an example implementation of adynamic vision system for dynamically learning an object concept aboutan object of interest according to some embodiments of the presentdisclosure.

FIG. 124 is a schematic illustrating an example architecture of adynamic vision system according to some embodiments of the presentdisclosure.

FIG. 125 is a flow diagram illustrating a method for object recognitionby a dynamic vision system according to some embodiments of the presentdisclosure.

FIG. 126 is a schematic illustrating an example implementation of adynamic vision system for modelling, simulating and optimizing variousoptical, mechanical, design and lighting parameters of the dynamicvision system according to some embodiments of the present disclosure.

FIG. 127 is a schematic view illustrating an example implementation of adynamic vision system depicting detailed view of various componentsalong with integration of the dynamic vision system with one or morethird party systems according to some embodiments of the presentdisclosure.

FIG. 128 is a schematic illustrating an example environment of a fleetmanagement platform according to some embodiments of the presentdisclosure.

FIG. 129 is a schematic illustrating example configurations of amulti-purpose robot and a special purpose robot according to someembodiments of the present disclosure.

FIG. 130 is a schematic illustrating an example platform-levelintelligence layer of a fleet management platform according to someembodiments of the present disclosure.

FIG. 131 is a schematic illustrating an example configuration of anintelligence layer according to some embodiments of the presentdisclosure.

FIG. 132 is a schematic illustrating an example security frameworkaccording to some embodiments of the present disclosure.

FIG. 133 is a schematic illustrating an example environment of a fleetmanagement platform according to some embodiments of the presentdisclosure.

FIG. 134 is a schematic illustrating an example data flow of a jobconfiguration system according to some embodiments of the presentdisclosure.

FIG. 135 is a schematic illustrating an example data flow of a fleetoperations system according to some embodiments of the presentdisclosure.

FIG. 136 is a schematic illustrating an example job parsing system andtask definition system and an example data flow thereof according tosome embodiments of the present disclosure.

FIG. 137 is a schematic illustrating an example fleet configurationsystem and an example data flow thereof according to some embodiments ofthe present disclosure.

FIG. 138 is a schematic illustrating an example workflow definitionsystem and an example data flow thereof according to some embodiments ofthe present disclosure.

FIG. 139 is a schematic illustrating example configurations of amulti-purpose robot and components thereof according to some embodimentsof the present disclosure.

FIG. 140 is a schematic illustrating an example architecture of therobot control system according to some embodiments of the presentdisclosure

FIG. 141 is a schematic illustrating an example architecture of therobot control system 12150 that utilizes data from multiple sensors inthe vision and sensing system according to some embodiments of thepresent disclosure.

FIG. 142 is a schematic illustrating an example vision and sensingsystem of a robot according to some embodiments of the presentdisclosure.

FIG. 143 is a schematic illustrating an example process that is executedby a multipurpose robot to harvest crops according to some embodimentsof the present disclosure.

FIG. 144 is a schematic illustrating an example environment of theintermodal smart container system according to some embodiments of thepresent disclosure.

FIG. 145 is a schematic illustrating example configurations of a smartcontainer according to some embodiments of the present disclosure.

FIG. 146 is a schematic illustrating an intelligence service adapted toprovide intelligence services to the smart intermodal container systemaccording to some embodiments of the present disclosure.

FIG. 147 is a schematic illustrating a digital twin module according tosome embodiments of the present disclosure according to some embodimentsof the present disclosure.

FIG. 148 illustrates an example embodiment of a method of receivingrequests to update one or more properties of digital twins of shippingentities and/or environments.

FIG. 149 illustrates an example embodiment of a method for updating aset of cost of downtime values in the digital twin of a smart containeraccording to some embodiments of the present disclosure.

FIG. 150 is a schematic illustrating an example environment of a digitalproduct network according to some embodiments of the present disclosure.

FIG. 151 is a schematic illustrating an example environment of aconnected product according to some embodiments of the presentdisclosure.

FIG. 152 is a schematic illustrating an example environment of a digitalproduct network according to some embodiments of the present disclosure.

FIG. 153 is a schematic illustrating an example environment of a digitalproduct network according to some embodiments of the present disclosure.

FIG. 154 is a flow diagram illustrating a method of using product leveldata according to some embodiments of the disclosure.

FIG. 155 is a schematic illustrating an example environment of a digitalproduct network according to some embodiments of the present disclosure.

FIG. 156 is a schematic illustrating an example of a smart futurescontract system according to some embodiments of the present disclosure.

FIG. 157 is a schematic illustrating an example environment of an edgenetworking system according to some embodiments of the presentdisclosure.

FIG. 158 is a schematic illustrating an example environment of an edgenetworking system including a VCN bus according to some embodiments ofthe present disclosure.

FIG. 159 a schematic illustrating an example environment of an edgenetworking system according to some embodiments of the presentdisclosure including a configured device EDNW system.

FIG. 160 is a schematic view of an exemplary embodiment of the quantumcomputing service according to some embodiments of the presentdisclosure.

FIG. 161 illustrates quantum computing service request handlingaccording to some embodiments of the present disclosure.

FIG. 162 is a diagrammatic view that illustrates embodiments of thebiology-based value chain network system in accordance with the presentdisclosure.

FIG. 163 is a diagrammatic view of the thalamus service and how itcoordinates within the modules in accordance with the presentdisclosure.

FIG. 164 is a block diagram showing an energy system that maycommunicate with similar systems, subsystems, components, and a valuechain network management platform according to some embodiments of thepresent disclosure.

FIG. 165 is a block diagram showing a schematic of a dual-processartificial neural network system according to some embodiments of thepresent disclosure.

FIG. 166A is a diagrammatic view that illustrates an example environmentof the distributed database system in accordance with the presentdisclosure.

FIG. 166B is a diagrammatic view that illustrates an examplearchitecture of the distributed database system in accordance with thepresent disclosure.

FIGS. 167A-167B are diagrammatic views that illustrate storage of datain the distributed database system in accordance with the presentdisclosure.

FIGS. 168A-168B are diagrammatic views that illustrate systems andmodules for implementing the distributed database system in accordancewith the present disclosure.

FIG. 169A-169B are process diagrams illustrating example methods forresponding to queries received by the distributed database system inaccordance with the present disclosure.

FIGS. 169C-169D are process diagrams illustrating example methods foroptimizing a dynamic ledger maintained by the distributed databasesystem in accordance with the present disclosure.

FIGS. 170A-170B are data flow diagrams that illustrate example datatable creation queries being processed by the distributed databasesystem in accordance with the present disclosure.

FIGS. 171A-171B are data flow diagrams that illustrate example selectqueries being processed by the distributed database system in accordancewith the present disclosure.

FIGS. 172A-172C are data flow diagrams that illustrate the operation ofexample distributed join queries in the distributed database system inaccordance with the present disclosure

DETAILED DESCRIPTION

Over time, companies have increasingly used technology solutions toimprove outcomes related to a traditional supply chain like the onedepicted in FIG. 1 , such as software systems for predicting andmanaging customer demand, RFID and asset tracking systems for trackinggoods as they move through the supply chain, navigation and routingsystems to improve the efficiency of route selection, and the like.However, some large trends have placed manufacturers, retailers andother businesses under increasing pressure to improve supply chainperformance. First, online and ecommerce operators, in particularAmazon™ have become the largest retail channels for many categories ofgoods and have introduced distribution and fulfillment centers 112throughout some geographies like the United States that house hundredsof thousands, and sometimes more, product categories (SKUs), so thatcustomers can receive items the day after they are ordered, and in somecases on the same day (and in some cases delivered to the door by adrone, robot, and/or autonomous vehicle. For retailers that do not haveextensive geographic distribution of fulfillment centers or warehouses,customer expectations for speed of delivery place increased pressure onsupply chain efficiency and optimization. Accordingly, a need stillexists for improved supply chain methods and systems.

Second, agile manufacturing capabilities (such as using 3D printing androbotic assembly techniques, among others), customer profilingtechnologies, and online ratings and reviews have led to increasedcustomer expectations for customization and personalization of products.Accordingly, in order to compete, manufacturers and retailers needimproved methods and systems for understanding, predicting, andsatisfying customer demand.

Historically, supply chain management and demand planning and managementhave been largely separate activities, unified primarily when demand isconverted to an order, which is passed to the supply side forfulfillment in a supply chain. As expectations for speed andpersonalization increase, a need exists for methods and systems that canprovide unified orchestration of supply and demand.

In parallel with these other large trends has been the emergence of theInternet of Things, in which some categories of products, particularlysmart home products like thermostats, lighting systems, and speakers,are increasingly enabled with onboard network connectivity andprocessing capability, often including a voice controlled intelligentagent like Alexa™ or Siri™ that allows device control and triggering ofcertain application features, such as playing music, or even ordering aproduct. In some cases, smart products 650 even initiate orders, such asprinters that order refill cartridges. Intelligent products 650 are insome cases involved in a coordinated system, such as where an Amazon™Echo™ product controls a television, or where a sensor-enabledthermostat or security camera connects to a mobile device, but mostintelligent products are still involved in sets of largely isolated,application-specific interactions. As artificial intelligencecapabilities increase, and as more and more computing and networkingpower is moved to network-enabled edge devices and systems that residein supply environments 670, in demand environments 672, and in all ofthe locations, systems, and facilities that populate the path of aproduct 1510 from the loading dock of a manufacturer to the point ofdestination 612 of a customer 662 or retailers 664, a need andopportunity exists for dramatically improved intelligence, control, andautomation of all of the factors involved in demand and supply.

Value Chain Networks

Referring to FIG. 2 , a block diagram is presented at 200 showingcomponents and interrelationships of systems and processes of a valuechain network. In example embodiments, “value chain network,” as usedherein, refers to elements and interconnections of historicallysegregated demand management systems and processes and supply chainmanagement systems and processes, enabled by the development andconvergence of numerous diverse technologies. In example embodiments avalue chain control tower 260 (e.g., referred to herein in some cases asa “value chain network management platform”, a “VCNP”, or simply as “thesystem”, or “the platform”) may be connected to, in communication with,or otherwise operatively coupled with data processing facilitiesincluding, but not limited to, big data centers (e.g., big dataprocessing 230) and related processing functionalities that receive dataflow, data pools, data streams and/or other data configurations andtransmission modalities received from, for example, digital productnetworks 21002, directly from customers (e.g., direct connected customer250), or some other third party 220. Communications related to marketorchestration activities and communications 210, analytics 232, or someother type of input may also be utilized by the value chain controltower for demand enhancement 262, synchronized planning 234, intelligentprocurement 238, dynamic fulfillment 240 or some other smart operationinformed by coordinated and adaptive intelligence, as described herein.

Referring to FIG. 3 , another block diagram is presented showingcomponents and interrelationships of systems and processes of a valuechain network and related uses cases, data handling, and associatedentities. In example embodiments, the value chain control tower 360 maycoordinate market orchestration activities 310 including, but notlimited to, demand curve management 352, synchronization of an ecosystem348, intelligent procurement 344, dynamic fulfillment 350, value chainanalytics 340, and/or smart supply chain operations 342. In exampleembodiments, the value chain control tower 360 may be connected to, incommunication with, or otherwise operatively coupled with adaptive datapipelines 302 and processing facilities that may be further connectedto, in communication with, or otherwise operationally coupled withexternal data sources 320 and a data handling stack 330 (e.g., valuechain network technology) that may include intelligent, user-adaptiveinterfaces, adaptive intelligence and control 332, and/or adaptive datamonitoring and storage 334, as described herein. The value chain controltower 302 may also be further connected to, in communication with, orotherwise operatively coupled with additional value chain entitiesincluding, but not limited to, digital product networks 21002, customers(e.g., directed connected customers 362), and/or other connectedoperations 364 and entities of a value chain network.

Digital Product Networks (“DPN”)

Referring to FIG. 4 , a block diagram is presented showing componentsand interrelationships of systems and processes of the digital productsnetworks at 400. In example embodiments, products (including goods andservices) may create and transmit data, such as product level data, to acommunication layer within the value chain network technology stackand/or to an edge data processing facility. This data may produceenhanced product level data and may be combined with third party datafor further processing, modeling or other adaptive or coordinatedintelligence activity, as described herein. This may include, but is notlimited to, producing and/or simulating product and value chain usecases, the data for which may be utilized by products, productdevelopment processes, product design, and the like.

Stack View Examples

Referring to FIG. 5 , a block diagram is presented at 500 showingcomponents and interrelationships of systems and processes of a valuechain network technology stack, which may include, but is not limited toa presentation layer, an intelligence layer, and serverlessfunctionalities such as platforms (e.g., development and hostingplatforms), data facilities (e.g., relating to data with IoT and BigData), and data aggregation facilities. In example embodiments, thepresentation layer may include, but is not limited to, a user interface,and modules for investigation and discovery and tracking users'experience and engagements. In example embodiments, the intelligencelayer may include, but is not limited to, a statistical and computationmethods, semantic models, an analytics library, a developmentenvironment for analytics, algorithms, logic and rules, and machinelearning. In example embodiments, the platforms or the value chainnetwork technology stack may include a development environment, APIs forconnectivity, cloud and/or hosting applications, and device discovery.In example embodiments, the data aggregation facilities or layer mayinclude, but is not limited to, modules for data normalization forcommon transmission and heterogeneous data collection from disparatedevices. In example embodiments, the data facilities or layer mayinclude, but is not limited to, IoT and big data access, control, andcollection and alternatives. In example embodiments, the value chainnetwork technology stack may be further associated with additional datasources and/or technology enablers.

Value Chain Orchestration from a Command Platform

FIG. 6 illustrates a connected value chain network 668 in which a valuechain network management platform 604 (referred to herein in some casesas a “value chain control tower,” the “VCNP,” or simply as “the system,”or “the platform”) orchestrates a variety of factors involved inplanning, monitoring, controlling, and optimizing various entities andactivities involved in the value chain network 668, such as supply andproduction factors, demand factors, logistics and distribution factors,and the like. By virtue of a unified platform 604 for monitoring andmanaging supply factors and demand factors as well as status information(e.g., quality and status, plan, order and confirm, and/or track andtrace) can be shared about and between various entities (e.g., includingcustomers/consumers, suppliers, distribution such as distributors,suppliers, and production such as producers or production facilities) asdemand factors are understood and accounted for, as orders are generatedand fulfilled, and as products are created and moved through a supplychain. The value chain network 668 may include not only an intelligentproduct 1510, but all of the equipment, infrastructure, personnel andother entities involved in planning and satisfying demand for it.

Value Chain Network and Value Chain Network Management Platform

Referring to FIG. 7 , the value chain network 668 managed by a valuechain management platform 604 may include a set of value chain networkentities 652, such as, without limitation: a product 1510, which may bean intelligent product 1510; a set of production facilities 674 involvedin producing finished goods, components, systems, sub-systems, materialsused in goods, or the like; various entities, activities and othersupply factors 648 involved in supply environments 670, such assuppliers 642, points of origin 610, and the like; various entities,activities and other demand factors 644 involved in demand environments672, such as customers 662 (including consumers, businesses, andintermediate customers such as value added resellers and distributors),retailers 664 (including online retailers, mobile retailers,conventional bricks and mortar retailers, pop-up shops and the like) andthe like located and/or operating at various destinations 612; variousdistribution environments 678 and distribution facilities 658, such aswarehousing facilities 654, fulfillment facilities 628, and deliverysystems 632, and the like, as well as maritime facilities 622, such asport infrastructure facilities 660, floating assets 620, and shipyards638, among others. In embodiments, the value chain network managementplatform 604 monitors, controls, and otherwise enables management (andin some cases autonomous or semi-autonomous behavior) of a wide range ofvalue chain network 668 processes, workflows, activities, events andapplications 630 (collectively referred to in some cases simply as“applications 630”).

Referring still to FIG. 7 , a high-level schematic of the value chainnetwork management platform 604 is illustrated. The value chain networkmanagement platform 604 may include a set of systems, applications,processes, modules, services, layers, devices, components, machines,products, sub-systems, interfaces, connections, and other elementsworking in coordination to enable intelligent management of a set ofvalue chain entities 652 that may occur, operate, transact or the likewithin, or own, operate, support or enable, one or more value chainnetwork processes, workflows, activities, events and/or applications 630or that may otherwise be part of, integrated with, linked to, oroperated on by the VCNP 604 in connection with a product 1510 (which maybe any category of product, such as a finished good, software product,hardware product, component product, material, item of equipment, itemof consumer packaged goods, consumer product, food product, beverageproduct, home product, business supply product, consumable product,pharmaceutical product, medical device product, technology product,entertainment product, or any other type of product and/or set ofrelated services, and which may, in embodiments, encompass anintelligent product 1510 that is enabled with a set of capabilities suchas, without limitation data processing, networking, sensing, autonomousoperation, intelligent agent, natural language processing, speechrecognition, voice recognition, touch interfaces, remote control,self-organization, self-healing, process automation, computation,artificial intelligence, analog or digital sensors, cameras, soundprocessing systems, data storage, data integration, and/or variousInternet of Things capabilities, among others.

In embodiments, the management platform 604 may include a set of datahandling layers 608 each of which is configured to provide a set ofcapabilities that facilitate development and deployment of intelligence,such as for facilitating automation, machine learning, applications ofartificial intelligence, intelligent transactions, state management,event management, process management, and many others, for a widevariety of value chain network applications and end uses. Inembodiments, the data handling layers 608 are configured in a topologythat facilitates shared data collection and distribution across multipleapplications and uses within the platform 604 by a value chainmonitoring systems layer 614. The value chain monitoring systems layer614 may include, integrate with, and/or cooperate with various datacollection and management systems 640, referred to for convenience insome cases as data collection systems 640, for collecting and organizingdata collected from or about value chain entities 652, as well as datacollected from or about the various data layers 624 or services orcomponents thereof. In embodiments, the data handling layers 608 areconfigured in a topology that facilitates shared or common data storageacross multiple applications and uses of the platform 604 by a valuechain network-oriented data storage systems layer 624, referred toherein for convenience in some cases simply as a data storage layer 624or storage layer 624. As shown in FIG. 7 , the data handling layers 608may also include an adaptive intelligent systems layer 614. The adaptiveintelligence systems layer 614 may include a set of data processing,artificial intelligence and computational systems 634 that are describedin more detail elsewhere throughout this disclosure. The dataprocessing, artificial intelligence and computational systems 634 mayrelate to artificial intelligence (e.g., expert systems, artificialintelligence, neural, supervised, machine learning, deep learning,model-based systems, and the like). Specifically, the data processing,artificial intelligence and computational systems 634 may relate tovarious examples, in some embodiments, such as use of a recurrentnetwork as adaptive intelligence system operating on a blockchain oftransactions in a supply chain to determine a pattern, use withbiological systems, opportunity mining (e.g., where artificialintelligence system may be used to monitor for new data sources asopportunities for automatically deploying intelligence), robotic processautomation (e.g., automation of intelligent agents for variousworkflows), edge and network intelligence (e.g., implicated onmonitoring systems such as adaptively using available RF spectrum,adaptively using available fixed network spectrum, adaptively storingdata based on available storage conditions, adaptively sensing based ona kind of contextual sensing), and the like.

In embodiments, the data handling layers 608 may be depicted in verticalstacks or ribbons in the figures and may represent many functionalitiesavailable to the platform 604 including storage, monitoring, andprocessing applications and resources and combinations thereof. Inembodiments, the set of capabilities of the data handling layers 608 mayinclude a shared microservices architecture. By way of these examples,the set of capabilities may be deployed to provide multiple distinctservices or applications, which can be configured as one or moreservices, workflows, or combinations thereof. In some examples, the setof capabilities may be deployed within or be resident to certainapplications or processes. In some examples, the set of capabilities caninclude one or more activities marshaled for the benefit of theplatform. In some examples, the set of capabilities may include one ormore events organized for the benefit of the platform. In embodiments,one of the sets of capabilities of the platform may be deployed withinat least a portion of a common architecture such as common architecturethat supports a common data schema. In embodiments, one of the sets ofcapabilities of the platform may be deployed within at least a portionof a common architecture that can support a common storage. Inembodiments, one of the sets of capabilities of the platform may bedeployed within at least a portion of a common architecture that cansupport common monitoring systems. In embodiments, one or more sets ofcapabilities of the platform may be deployed within at least a portionof a common architecture that can support one or more common processingframeworks. In embodiments, the set of capabilities of the data handlinglayers 608 can include examples where the storage functionality supportsscalable processing capabilities, scalable monitoring systems, digitaltwin systems, payments interface systems, and the like. By way of theseexamples, one or more software development kits can be provided by theplatform along with deployment interfaces to facilitate connections anduse of the capabilities of the data handling layers 608. In furtherexamples, adaptive intelligence systems may analyze, learn, configure,and reconfigure one or more of the capabilities of the data handlinglayers 608. In embodiments, the platform 604 may, for example, include acommon data storage schema serving a shipyard entity related service anda warehousing entity service. There are many other applicable examplesand combinations applicable to the foregoing example including the manyvalue chain entities disclosed herein. By way of these examples, theplatform 604 may be shown to create connectivity (e.g., supply ofcapabilities and information) across many value chain entities. In manyexamples, there are pairings (doubles, triples, quadruplets, etc.) ofsimilar kinds of value chain entities using one or more smaller sets ofcapabilities of the data handling layers 608 to deploy (interact with,rely on, etc.) a common data schema, a common architecture, a commoninterface, and the like. While services and capabilities can be providedto single value chain entities, the platform can be shown to providemyriad benefits to value chains and consumers by supporting connectivityacross value chain entities and applications used by the entities.

Value Chain Network Entities Managed by the Platform

Referring to FIG. 8 , the value chain network management platform 604 isillustrated in connection with a set of value chain entities 652 thatmay be subject to management by the platform 604, may integrate with orinto the platform 604, and/or may supply inputs to and/or take outputsfrom the platform 604, such as ones involved in or for a wide range ofvalue chain activities (such as supply chain activities, logisticsactivities, demand management and planning activities, deliveryactivities, shipping activities, warehousing activities, distributionand fulfillment activities, inventory aggregation, storage andmanagement activities, marketing activities, and many others, asinvolved in various value chain network processes, workflows,activities, events and applications 630 (collectively “applications 630”or simply “activities”)). Connections with the value chain entities 652may be facilitated by a set of connectivity facilities 642 andinterfaces 702, including a wide range of components and systemsdescribed throughout this disclosure and in greater detail below. Thismay include connectivity and interface capabilities for individualservices of the platform, for the data handling layers, for the platformas a whole, and/or among value chain entities 652, among others.

These value chain entities 652 may include any of the wide variety ofassets, systems, devices, machines, components, equipment, facilities,individuals or other entities mentioned throughout this disclosure or inthe documents incorporated herein by reference, such as, withoutlimitation: machines 724 and their components (e.g., delivery vehicles,forklifts, conveyors, loading machines, cranes, lifts, haulers, trucks,loading machines, unloading machines, packing machines, pickingmachines, and many others, including robotic systems, e.g., physicalrobots, collaborative robots (e.g., “cobots”), drones, autonomousvehicles, software bots and many others); products 650 (which may be anycategory of products, such as a finished goods, software products,hardware products, component products, material, items of equipment,items of consumer packaged goods, consumer products, food products,beverage products, home products, business supply products, consumableproducts, pharmaceutical products, medical device products, technologyproducts, entertainment products, or any other type of products and/orset of related services); value chain processes 722 (such as shippingprocesses, hauling processes, maritime processes, inspection processes,hauling processes, loading/unloading processes, packing/unpackingprocesses, configuration processes, assembly processes, installationprocesses, quality control processes, environmental control processes(e.g., temperature control, humidity control, pressure control,vibration control, and others), border control processes, port-relatedprocesses, software processes (including applications, programs,services, and others), packing and loading processes, financialprocesses (e.g., insurance processes, reporting processes, transactionalprocesses, and many others), testing and diagnostic processes, securityprocesses, safety processes, reporting processes, asset trackingprocesses, and many others); wearable and portable devices 720 (such asmobile phones, tablets, dedicated portable devices for value chainapplications and processes, data collectors (including mobile datacollectors), sensor-based devices, watches, glasses, hearables,head-worn devices, clothing-integrated devices, arm bands, bracelets,neck-worn devices, AR/VR devices, headphones, and many others); workers718 (such as delivery workers, shipping workers, barge workers, portworkers, dock workers, train workers, ship workers, distribution offulfillment center workers, warehouse workers, vehicle drivers, businessmanagers, engineers, floor managers, demand managers, marketingmanagers, inventory managers, supply chain managers, cargo handlingworkers, inspectors, delivery personnel, environmental control managers,financial asset managers, process supervisors and workers (for any ofthe processes mentioned herein), security personnel, safety personneland many others); suppliers 642 (such as suppliers of goods and relatedservices of all types, component suppliers, ingredient suppliers,materials suppliers, manufacturers, and many others); customers 662(including consumers, licensees, businesses, enterprises, value addedand other resellers, retailers, end users, distributors, and others whomay purchase, license, or otherwise use a category of goods and/orrelated services); a wide range of operating facilities 712 (such asloading and unloading docks, storage and warehousing facilities 654,vaults, distribution facilities 658 and fulfillment centers 628, airtravel facilities 740 (including aircraft, airports, hangars, runways,refueling depots, and the like), maritime facilities 622 (such as portinfrastructure facilities 622 (such as docks, yards, cranes,roll-on/roll-off facilities, ramps, containers, container handlingsystems, waterways 732, locks, and many others), shipyard facilities638, floating assets 620 (such as ships, barges, boats and others),facilities and other items at points of origin 610 and/or points ofdestination 628, hauling facilities 710 (such as container ships,barges, and other floating assets 620, as well as land-based vehiclesand other delivery systems 632 used for conveying goods, such as trucks,trains, and the like); items or elements factoring in demand (i.e.,demand factors 644) (including market factors, events, and many others);items or elements factoring in supply (i.e., supply factors648)(including market factors, weather, availability of components andmaterials, and many others); logistics factors 750 (such as availabilityof travel routes, weather, fuel prices, regulatory factors, availabilityof space (such as on a vehicle, in a container, in a package, in awarehouse, in a fulfillment center, on a shelf, or the like), and manyothers); retailers 664 (including online retailers 730 and others suchas in the form of eCommerce sites 730); pathways for conveyance (such aswaterways 732, roadways 734, air travel routes, railways 738 and thelike); robotic systems 744 (including mobile robots, cobots, roboticsystems for assisting human workers, robotic delivery systems, andothers); drones 748 (including for package delivery, site mapping,monitoring or inspection, and the like); autonomous vehicles 742 (suchas for package delivery); software platforms 752 (such as enterpriseresource planning platforms, customer relationship management platforms,sales and marketing platforms, asset management platforms, Internet ofThings platforms, supply chain management platforms, platform as aservice platforms, infrastructure as a service platforms, software-baseddata storage platforms, analytic platforms, artificial intelligenceplatforms, and others); and many others. In some example embodiments,the product 1510 may be encompassed as an intelligent product 1510 orthe VCNP 604 may include the intelligent product 1510. The intelligentproduct 1510 may be enabled with a set of capabilities such as, withoutlimitation data processing, networking, sensing, autonomous operation,intelligent agent, natural language processing, speech recognition,voice recognition, touch interfaces, remote control, self-organization,self-healing, process automation, computation, artificial intelligence,analog or digital sensors, cameras, sound processing systems, datastorage, data integration, and/or various Internet of Thingscapabilities, among others. The intelligent product 1510 may include aform of information technology. The intelligent product 1510 may have aprocessor, computer random access memory, and a communication module.The intelligent product 1510 may be a passive intelligent product thatis similar to a RFID type of data structure where the intelligentproduct may be pinged or read. The product 1510 may be considered avalue chain network entity (e.g., under control of platform) and may berendered intelligent by surrounding infrastructure and adding an RFIDsuch that data may be read from the intelligent product 1510. Theintelligent product 1510 may fit in a value chain network in a connectedway such that connectivity was built around the intelligent product 1510through a sensor, an IoT device, a tag, or another component.

In embodiments, the monitoring systems layer 614 may monitor any or allof the value chain entities 652 in a value chain network 668, mayexchange data with the value chain entities 652, may provide controlinstructions to or take instructions from any of the value chainentities 652, or the like, such as through the various capabilities ofthe data handling layers 608 described throughout this disclosure.

Network Characteristics of the Value Chain Network Entities

Referring to FIG. 9 , orchestration of a set of deeply interconnectedvalue chain network entities 652 in a value chain network 668 by thevalue chain network management platform 604 is illustrated. Each of thevalue chain network entities 652 may have a connection to the VCNP 604,to a set of other value chain network entities 652 (which may be a localnetwork connection, a peer-to-peer connection, a mobile networkconnection, a connection via a cloud, or other connection), and/orthrough the VCNP 604 to other value chain network entities 652. Thevalue chain network management platform 604 may manage the connections,configure or provision resources to enable connectivity, and/or manageapplications 630 that take advantage of the connections, such as byusing information from one set of entities 652 to inform applications630 involving another set of entities 652, by coordinating activities ofa set of entities 652, by providing input to an artificial intelligencesystem of the VCNP 604 or of or about a set of entities 652, byinteracting with edge computation systems deployed on or in entities 652and their environments, and the like.

The entities 652 may be external such that the VCNP 604 may interactwith these entities 652. When the VCNP 604 functions as the controltower to establish monitoring (e.g., establish monitoring such as commonmonitoring across several entities 652). In one unified platform, theremay be an interface where a user may view various items such as user'sdestinations, ports, air and rail assets, as well as orders, etc. Then,the next step may be to establish a common data schema that enablesservices that work on or in any one of these applications. This mayinvolve taking any of the data that is flowing through or about any ofthese entities 652 and pull the data into a framework where otherapplications across supply and demand may interact with the entities652. This may be a shared data pipeline coming from an IoT system andother external data sources, feeding into the monitoring layer, beingstored in a common data schema in the storage layer, and then variousintelligence may be trained to identify implications across theseentities 652. In an example embodiment, a supplier may be bankrupt, or adetermination is made that the supplier is bankrupt, and then the VCNP604 may automatically trigger a substitute smart contract to be sent toa secondary supplier with altered terms. There may be management ofdifferent aspects of the supply chain. For example, changing pricinginstantly and automatically on the demand side in response to one moresupplier's being identified as bankrupt (e.g., from bankruptcyannouncement). Other similar examples may be used based on what occursin that automation layer which may be enabled by the VCNP 604. Then, atthe interface layer of this VCNP 604, a digital twin may be used by userto view all these entities 652 that are not typically shown together andmonitor what is going on with each of these entities 652 includingidentification of problem states. For example, after viewing threequarters of bad financial reports on a supplier, a report may be flaggedto watch it closely for potential future bankruptcy, etc.

For example, an IoT system deployed in a fulfillment center 628 maycoordinate with an intelligent product 1510 that takes customer feedbackabout the product 1510, and an application 630 for the fulfillmentcenter 628 may, upon receiving customer feedback via a connection pathto the intelligent product 1510 about a problem with the product 1510,initiate a workflow to perform corrective actions on similar products650 before the products 650 are sent out from the fulfillment center628. Similarly, a port infrastructure facility 660, such as a yard forholding shipping containers, may inform a fleet of floating assets 620via connections to the floating assets 620 (such as ships, barges, orthe like) that the port is near capacity, thereby kicking off anegotiation process (which may include an automated negotiation based ona set of rules and governed by a smart contract) for the remainingcapacity and enabling some assets 620 to be redirected to alternativeports or holding facilities. These and many other connections amongvalue chain network entities 652, whether one-to-one connections,one-to-many connections, many-to-many connections, or connections amongdefined groups of entities 652 (such as ones controlled by the sameowner or operator), are encompassed herein as applications 630 managedby the VCNP 604.

Value Chain Network Activities and Applications Managed by the Platform

Referring to FIG. 10 , the set of applications 614 provided on the VCNP604, integrated with the VCNP 604 and/or managed by or for the VCNP 604and/or involving a set of value chain network entities 652 may include,without limitation, one or more of any of a wide range of types ofapplications, such as: a supply chain management applications 21004(such as, without limitation, for management of timing, quantities,logistics, shipping, delivery, and other details of orders for goods,components, and other items); an asset management application 814 (suchas, without limitation, for managing value chain assets, such asfloating assets (such as ships, boats, barges, and floating platforms),real property (such as used for location of warehouses, ports,shipyards, distribution centers and other buildings), equipment,machines and fixtures (such as used for handling containers, cargo,packages, goods, and other items), vehicles (such as forklifts, deliverytrucks, autonomous vehicles, and other systems used to move items),human resources (such as workers), software, information technologyresources, data processing resources, data storage resources, powergeneration and/or storage resources, computational resources and otherassets); a finance application 822 (such as, without limitation, forhandling finance matters relating to value chain entities and assets,such as involving payments, security, collateral, bonds, customs,duties, imposts, taxes and others); a 6 (such as, without limitation,for managing risk or liability with respect to a shipment, goods, aproduct, an asset, a person, a floating asset, a vehicle, an item ofequipment, a component, an information technology system, a securitysystem, a security event, a cybersecurity system, an item of property, ahealth condition, mortality, fire, flood, weather, disability,negligence, business interruption, injury, damage to property, damage toa business, breach of a contract, and others); a demand managementapplication 824 (such as, without limitation, an application foranalyzing, planning, or promoting interest by customers of a category ofgoods that can be supplied by or with facilities of a value chainproduct or service, such as a demand planning application, a demandprediction application, a sales application, a future demand aggregationapplication, a marketing application, an advertising application, ane-commerce application, a marketing analytics application, a customerrelationship management application, a search engine optimizationapplication, a sales management application, an advertising networkapplication, a behavioral tracking application, a marketing analyticsapplication, a location-based product or service-targeting application,a collaborative filtering application, a recommendation engine for aproduct or service, and others, including ones that use or are enabledby one or more features of an intelligent product 1510 or that areexecuted using intelligence capabilities on an intelligent product1510); a trading application 858 (such as, without limitation, a buyingapplication, a selling application, a bidding application, an auctionapplication, a reverse auction application, a bid/ask matchingapplication, an analytic application for analyzing value chainperformance, yield, return on investment, or other metrics, or others);a tax application 850 (such as, without limitation, for managing,calculating, reporting, optimizing, or otherwise handling data, events,workflows, or other factors relating to a tax, a tariff, an impost, alevy, a tariff, a duty, a credit, a fee or other government-imposedcharge, such as, without limitation, customs duties, value added tax,sales tax, income tax, property tax, municipal fees, pollution tax,renewal energy credit, pollution abatement credit, import duties, exportduties, and others); an identity management application 830 (such as formanaging one or more identities of entities 652 involved in a valuechain, such as, without limitation, one or more of an identityverification application, a biometric identify validation application, apattern-based identity verification application, a location-basedidentity verification application, a user behavior-based application, afraud detection application, a network address-based fraud detectionapplication, a black list application, a white list application, acontent inspection-based fraud detection application, or other frauddetection application; an inventory management application 820 (such as,without limitation, for managing inventory in a fulfillment center,distribution center, warehouse, storage facility, store, port, ship orother floating asset, or other location); a security application,solution or service 834 (referred to herein as a security application,such as, without limitation, any of the identity management applications830 noted above, as well as a physical security system (such as for anaccess control system (such as using biometric access controls,fingerprinting, retinal scanning, passwords, and other access controls),a safe, a vault, a cage, a safe room, a secure storage facility, or thelike), a monitoring system (such as using cameras, motion sensors,infrared sensors and other sensors), a perimeter security system, afloating security system for a floating asset, a cyber security system(such as for virus detection and remediation, intrusion detection andremediation, spam detection and remediation, phishing detection andremediation, social engineering detection and remediation, cyber-attackdetection and remediation, packet inspection, traffic inspection, DNSattack remediation and detection, and others) or other securityapplication); a safety application 840 (such as, without limitation, forimproving safety of workers, for reducing the likelihood of damage toproperty, for reducing accident risk, for reducing the likelihood ofdamage to goods (such as cargo), for risk management with respected toinsured items, collateral for loans, or the like, including anyapplication for detecting, characterizing or predicting the likelihoodand/or scope of an accident or other damaging event, including safetymanagement based on any of the data sources, events or entities notedthroughout this disclosure or the documents incorporated herein byreference); a blockchain application 844 (such as, without limitation, adistributed ledger capturing a series of transactions, such as debits orcredits, purchases or sales, exchanges of in kind consideration, smartcontract events, or the like, or other blockchain-based application); afacility management application 850 (such as, without limitation, formanaging infrastructure, buildings, systems, real property, personalproperty, and other property involved in supporting a value chain, suchas a shipyard, a port, a distribution center, a warehouse, a dock, astore, a fulfillment center, a storage facility, or others, as well asfor design, management or control of systems and facilities in or arounda property, such as an information technology system, arobotic/autonomous vehicle system, a packaging system, a packing system,a picking system, an inventory tracking system, an inspection system, arouting system for mobile robots, a workflow system for human assets, orthe like); a regulatory application 852 (such as, without limitation, anapplication for regulating any of the applications, services,transactions, activities, workflows, events, entities, or other itemsnoted herein and in the documents incorporated by reference herein, suchas regulation of permitted routes, permitted cargo and goods, permittedparties to transactions, required disclosures, privacy, pricing,marketing, offering of goods and services, use of data (including dataprivacy regulations, regulations relating to storage of data andothers), banking, marketing, sales, financial planning, and manyothers); a commerce application, solution or service 854 (such as,without limitation an e-commerce site marketplace, an online site, anauction site or marketplace, a physical goods marketplace, anadvertising marketplace, a reverse-auction marketplace, an advertisingnetwork, or other marketplace); a vendor management application 832(such as, without limitation, an application for managing a set ofvendors or prospective vendors and/or for managing procurement of a setof goods, components or materials that may be supplied in a value chain,such as involving features such as vendor qualification, vendor rating,requests for proposal, requests for information, bonds or otherassurances of performance, contract management, and others); ananalytics application 838 (such as, without limitation, an analyticapplication with respect to any of the data types, applications, events,workflows, or entities mentioned throughout this disclosure or thedocuments incorporated by reference herein, such as a big dataapplication, a user behavior application, a prediction application, aclassification application, a dashboard, a pattern recognitionapplication, an econometric application, a financial yield application,a return on investment application, a scenario planning application, adecision support application, a demand prediction application, a demandplanning application, a route planning application, a weather predictionapplication, and many others); a pricing application 842 (such as,without limitation, for pricing of goods, services (including anymentioned throughout this disclosure and the documents incorporated byreference herein; and a smart contract application, solution, or service(referred to collectively herein as a smart contract application 848,such as, without limitation, any of the smart contract types referred toin this disclosure or in the documents incorporated herein by reference,such as a smart contract for sale of goods, a smart contract for anorder for goods, a smart contract for a shipping resource, a smartcontract for a worker, a smart contract for delivery of goods, a smartcontract for installation of goods, a smart contract using a token orcryptocurrency for consideration, a smart contract that vests a right,an option, a future, or an interest based on a future condition, a smartcontract for a security, commodity, future, option, derivative, or thelike, a smart contract for current or future resources, a smart contractthat is configured to account for or accommodate a tax, regulatory orcompliance parameter, a smart contract that is configured to execute anarbitrage transaction, or many others). Thus, the value chain managementplatform 604 may host an enable interaction among a wide range ofdisparate applications 630 (such term including the above-referenced andother value chain applications, services, solutions, and the like), suchthat by virtue of shared microservices, shared data infrastructure, andshared intelligence, any pair or larger combination or permutation ofsuch services may be improved relative to an isolated application of thesame type.

Referring still to FIG. 10 , the set of applications 614 provided on theVCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP604 and/or involving a set of value chain network entities 652 mayfurther include, without limitation: a payments application 860 (such asfor calculating payments (including based on situational factors such asapplicable taxes, duties and the like for the geography of an entity652), transferring funds, resolving payments to parties, and the like,for any of the applications 630 noted herein); a process managementapplication 862 (such as for managing any of the processes or workflowsdescribed throughout this disclosure, including supply processes, demandprocesses, logistics processes, delivery processes, fulfillmentprocesses, distribution processes, ordering processes, navigationprocesses, and many others); a compatibility testing application 864,such as for assessing compatibility among value chain network entities652 or activities involved in any of the processes, workflows,activities, or other applications 630 described herein (such as fordetermining compatibility of a container or package with a product 1510,the compatibility of a product 1510 with a set of customer requirements,the compatibility of a product 1510 with another product 1510 (such aswhere one is a refill, resupply, replacement part, or the like for theother), the compatibility of a infrastructure and equipment entities 652(such as between a container ship or barge and a port or waterway,between a container and a storage facility, between a truck and aroadway, between a drone or robot and a package, between a drone, AV orrobot and a delivery destination, and many others); an infrastructuretesting application 802 (such as for testing the capabilities ofinfrastructure elements to support a product 1510 or an application 630(such as, without limitation, storage capabilities, liftingcapabilities, moving capabilities, storage capacity, networkcapabilities, environmental control capabilities, software capabilities,security capabilities, and many others)); and/or an incident managementapplication 910 (such as for managing events, accidents, and otherincidents that may occur in one or more environments involving valuechain network entities 652, such as, without limitation, vehicleaccidents, worker injuries, shutdown incidents, property damageincidents, product damage incidents, product liability incidents,regulatory non-compliance incidents, health and/or safety incidents,traffic congestion and/or delay incidents (including network traffic,data traffic, vehicle traffic, maritime traffic, human worker traffic,and others, as well as combinations among them), product failureincidents, system failure incidents, system performance incidents, fraudincidents, misuse incidents, unauthorized use incidents, and manyothers).

Referring still to FIG. 10 , the set of applications 614 provided on theVCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP604 and/or involving a set of value chain network entities 652 mayfurther include, without limitation: a predictive maintenanceapplication 910 (such as for anticipating, predicting, and undertakingactions to manage faults, failures, shutdowns, damage, requiredmaintenance, required repairs, required service, required support, orthe like for a set of value chain network entities 652, such as products650, equipment, infrastructure, buildings, vehicles, and others); alogistics application 912 (such as for managing logistics for pickups,deliveries, transfer of goods onto hauling facilities, loading,unloading, packing, picking, shipping, driving, and other activitiesinvolving in the scheduling and management of the movement of products650 and other items between points of origin and points of destinationthrough various intermediate locations; a reverse logistic application914 (such as for handling logistics for returned products 650, wasteproducts, damaged goods, or other items that can be transferred on areturn logistics path); a waste reduction application 920 (such as forreducing packaging waste, solid waste, waste of energy, liquid waste,pollution, contaminants, waste of computing resources, waste of humanresources, or other waste involving a value chain network entity 652 oractivity); an augmented reality, mixed reality and/or virtual realityapplication 930 (such as for visualizing one or more value chain networkentities 652 or activities involved in one or more of the applications630, such as, without limitation, movement of a product 1510, theinterior of a facility, the status or condition of an item of goods, oneor more environmental conditions, a weather condition, a packingconfiguration for a container or a set of containers, or many others); ademand prediction application 940 (such as for predicting demand for aproduct 1510, a category of products, a potential product, and/or afactor involved in demand, such as a market factor, a wealth factor, ademographic factor, a weather factor, an economic factor, or the like);a demand aggregation application 942 (such as for aggregatinginformation, orders and/or commitments (optionally embodied in one ormore contracts, which may be smart contracts) for one or more products650, categories, or the like, including current demand for existingproducts and future demand for products that are not yet available); acustomer profiling application 944 (such as for profiling one or moredemographic, psychographic, behavioral, economic, geographic, or otherattributes of a set of customers, including based on historicalpurchasing data, loyalty program data, behavioral tracking data(including data captured in interactions by a customer with a smartproduct 1510), online clickstream data, interactions with intelligentagents, and other data sources); and/or a component supply application948 (such as for managing a supply chain of components for a set ofproducts 650).

Referring still to FIG. 10 , the set of applications 614 provided on theVCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP604 and/or involving a set of value chain network entities 652 mayfurther include, without limitation: a policy management application 868(such as for deploying one or more policies, rules, or the like forgovernance of one or more value chain network entities 652 orapplications 630, such as to govern execution of one or more workflows(which may involve configuring polices in the platform 604 on aper-workflow basis), to govern compliance with regulations (includingmaritime, food and drug, medical, environmental, health, safety, tax,financial reporting, commercial, and other regulations as describedthroughout this disclosure or as would be understood in the art), togovern provisioning of resources (such as connectivity, computing,human, energy, and other resources), to govern compliance with corporatepolicies, to govern compliance with contracts (including smartcontracts, wherein the platform 604 may automatically deploy governancefeatures to relevant entities 652 and applications 630, such as viaconnectivity facilities 642), to govern interactions with other entities(such as involving policies for sharing of information and access toresources), to govern data access (including privacy data, operationaldata, status data, and many other data types), to govern security accessto infrastructure, products, equipment, locations, or the like, and manyothers; a product configuration application 870 (such as for allowing aproduct manager and/or automated product configuration process(optionally using robotic process automation) to determine aconfiguration for a product 1510, including configuration on-the-fly,such as during agile manufacturing, or involving configuration orcustomization in route (such as by 3D printing one or more features orelements), or involving configuration or customization remotely, such asby downloading firmware, configuring field programmable gate arrays,installing software, or the like; a warehousing and fulfillmentapplication 872 (such as for managing a warehouse, distribution center,fulfillment center, or the like, such as involving selection ofproducts, configuring storage locations for products, determining routesby which personnel, mobile robots, and the like move products around afacility, determining picking and packing schedules, routes andworkflows, managing operations of robots, drones, conveyors, and otherfacilities, determining schedules for moving products out to loadingdocks or the like, and many other functions); a kit configuration anddeployment application 874 (such as for enabling a user of the VCNP toconfigure a kit, box, or otherwise pre-integrated, pre-provisioned,and/or pre-configured system to allow a customer or worker to rapidlydeploy a subset of capabilities of the VCNP 604 for a specific valuechain network entity 652 and/or application 630); and/or a producttesting application 878 for testing a product 1510 (including testingfor performance, activation of capabilities and features, safety,compliance with policy or regulations, quality, quality of service,likelihood of failure, and many other factors).

Referring still to FIG. 10 , the set of applications 614 provided on theVCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP604 and/or involving a set of value chain network entities 652 mayfurther include, without limitation a maritime fleet managementapplication 880 (for managing a set of maritime assets, such ascontainer ships, barges, boats, and the like, as well as relatedinfrastructure facilities such as docks, cranes, ports, and others, suchas to determine optimal routes for fleet assets based on weather,market, traffic, and other conditions, to ensure compliance withpolicies and regulations, to ensure safety, to improve environmentalfactors, to improve financial metrics, and many others); a shippingmanagement application 882 (such as for managing a set of shippingassets, such as trucks, trains, airplanes, and the like, such as tooptimize financial yield, to improve safety, to reduce energyconsumption, to reduce delays, to mitigate environmental impact, and formany other purposes); an opportunity matching application 884 (such asfor matching one or more demand factors with one or more supply factors,for matching needs and capabilities of value chain network entities 652,for identifying reverse logistics opportunities, for identifyingopportunities for inputs to enrich analytics, artificial intelligenceand/or automation, for identifying cost-saving opportunities, foridentifying profit and/or arbitrage opportunities, and many others); aworkforce management application 888 (such as for managing workers invarious work forces, including work forces in, on or for fulfillmentcenters, ships, ports, warehouses, distribution centers, enterprisemanagement locations, retail stores, online/ecommerce site managementfacilities, ports, ships, boats, barges, trains, depots, and otherfacilities mentioned throughout this disclosure); a distribution anddelivery application 890 (such as for planning, scheduling, routing, andotherwise managing distribution and delivery of products 650 and otheritems); and/or an enterprise resource planning (ERP) application 892(such as for planning utilization of enterprise resources, includingworkforce resources, financial resources, energy resources, physicalassets, digital assets, and other resources).

Core Capabilities and Interactions of the Data Handling Layers (AdaptiveIntelligence, Monitoring, Data Storage and Applications)

Referring to FIG. 11 , a high-level schematic of an embodiment of thevalue chain network management platform 604 is illustrated, including aset of systems, applications, processes, modules, services, layers,devices, components, machines, products, sub-systems, interfaces,connections, and other elements working in coordination to enableintelligent management of sets of the value chain entities 652 that mayoccur, operate, transact or the like within, or own, operate, support orenable, one or more value chain network processes, workflows,activities, events and/or applications 630 or that may otherwise be partof, integrated with, linked to, or operated on by the platform 604 inconnection with a product 1510 (which may be a finished good, softwareproduct, hardware product, component product, material, item ofequipment, consumer packaged good, consumer product, food product,beverage product, home product, business supply product, consumableproduct, pharmaceutical product, medical device product, technologyproduct, entertainment product, or any other type of product or relatedservice, which may, in embodiments, encompass an intelligent productthat is enabled with processing, networking, sensing, computation,and/or other Internet of Things capabilities). Value chain entities 652,such as involved in or for a wide range of value chain activities (suchas supply chain activities, logistics activities, demand management andplanning activities, delivery activities, shipping activities,warehousing activities, distribution and fulfillment activities,inventory aggregation, storage and management activities, marketingactivities, and many others, as involved in various value chain networkprocesses, workflows, activities, events and applications 630 mayinclude any of the wide variety of assets, systems, devices, machines,components, equipment, facilities, individuals or other entitiesmentioned throughout this disclosure or in the documents incorporatedherein by reference.

In embodiments, the value chain network management platform 604 mayinclude the set of data handling layers 608, each of which is configuredto provide a set of capabilities that facilitate development anddeployment of intelligence, such as for facilitating automation, machinelearning, applications of artificial intelligence, intelligenttransactions, intelligent operations, remote control, analytics,monitoring, reporting, state management, event management, processmanagement, and many others, for a wide variety of value chain networkapplications and end uses. In embodiments, the data handling layers 608may include a value chain network monitoring systems layer 614, a valuechain network entity-oriented data storage systems layer 624 (referredto in some cases herein for convenience simply as a data storage layer624), an adaptive intelligent systems layer 614 and a value chainnetwork management platform 604. The value chain network managementplatform 604 may include the data handling layers 608 such that thevalue chain network management platform 604 may provide management ofthe value chain network management platform 604 and/or management of theother layers such as the value chain network monitoring systems layer614, the value chain network entity-oriented data storage systems layer624 (e.g., data storage layer 624), and the adaptive intelligent systemslayer 614. Each of the data handling layers 608 may include a variety ofservices, programs, applications, workflows, systems, components andmodules, as further described herein and in the documents incorporatedherein by reference. In embodiments, each of the data handling layers608 (and optionally the platform 604 as a whole) is configured such thatone or more of its elements can be accessed as a service by other layers624 or by other systems (e.g., being configured as aplatform-as-a-service deployed on a set of cloud infrastructurecomponents in a microservices architecture). For example, the platform604 may have (or may configure and/or provision), and a data handlinglayer 608 may use, a set of connectivity facilities 642, such as networkconnections (including various configurations, types and protocols),interfaces, ports, application programming interfaces (APIs), brokers,services, connectors, wired or wireless communication links,human-accessible interfaces, software interfaces, micro-services, SaaSinterfaces, PaaS interfaces, IaaS interfaces, cloud capabilities, or thelike by which data or information may be exchanged between a datahandling layer 608 and other layers, systems or sub-systems of theplatform 604, as well as with other systems, such as value chainentities 652 or external systems, such as cloud-based or on-premisesenterprise systems (e.g., accounting systems, resource managementsystems, CRM systems, supply chain management systems and many others).Each of the data handling layers 608 may include a set of services(e.g., microservices), for data handling, including facilities for dataextraction, transformation and loading; data cleansing and deduplicationfacilities; data normalization facilities; data synchronizationfacilities; data security facilities; computational facilities (e.g.,for performing pre-defined calculation operations on data streams andproviding an output stream); compression and de-compression facilities;analytic facilities (such as providing automated production of datavisualizations) and others.

In embodiments, each data handling layer 608 has a set of applicationprogramming connectivity facilities 642 for automating data exchangewith each of the other data handling layers 608. These may include dataintegration capabilities, such as for extracting, transforming, loading,normalizing, compression, decompressing, encoding, decoding, andotherwise processing data packets, signals, and other information as itexchanged among the layers and/or the applications 630, such astransforming data from one format or protocol to another as needed inorder for one layer to consume output from another. In embodiments, thedata handling layers 608 are configured in a topology that facilitatesshared data collection and distribution across multiple applications anduses within the platform 604 by the value chain monitoring systems layer614. The value chain monitoring systems layer 614 may include, integratewith, and/or cooperate with various data collection and managementsystems 640, referred to for convenience in some cases as datacollection systems 640, for collecting and organizing data collectedfrom or about value chain entities 652, as well as data collected fromor about the various data layers 624 or services or components thereof.For example, a stream of physiological data from a wearable device wornby a worker undertaking a task or a consumer engaged in an activity canbe distributed via the monitoring systems layer 614 to multiple distinctapplications in the value chain management platform 604, such as onethat facilitates monitoring the physiological, psychological,performance level, attention, or other state of a worker and anotherthat facilitates operational efficiency and/or effectiveness. Inembodiments, the monitoring systems layer 614 facilitates alignment,such as time-synchronization, normalization, or the like of data that iscollected with respect to one or more value chain network entities 652.For example, one or more video streams or other sensor data collected ofor with respect to a worker 718 or other entity in a value chain networkfacility or environment, such as from a set of camera-enabled IoTdevices, may be aligned with a common clock, so that the relative timingof a set of videos or other data can be understood by systems that mayprocess the videos, such as machine learning systems that operate onimages in the videos, on changes between images in different frames ofthe video, or the like. In such an example, the monitoring systems layer614 may further align a set of videos, camera images, sensor data, orthe like, with other data, such as a stream of data from wearabledevices, a stream of data produced by value chain network systems (suchas ships, lifts, vehicles, containers, cargo handling systems, packingsystems, delivery systems, drones/robots, and the like), a stream ofdata collected by mobile data collectors, and the like. Configuration ofthe monitoring systems layer 614 as a common platform, or set ofmicroservices, that are accessed across many applications, maydramatically reduce the number of interconnections required by an owneror other operator within a value chain network in order to have agrowing set of applications monitoring a growing set of IoT devices andother systems and devices that are under its control.

In embodiments, the data handling layers 608 are configured in atopology that facilitates shared or common data storage across multipleapplications and uses of the platform 604 by the value chainnetwork-oriented data storage systems layer 624, referred to herein forconvenience in some cases simply as the data storage layer 624 orstorage layer 624. For example, various data collected about the valuechain entities 652, as well as data produced by the other data handlinglayers 608, may be stored in the data storage layer 624, such that anyof the services, applications, programs, or the like of the various datahandling layers 608 can access a common data source (which may comprisea single logical data source that is distributed across disparatephysical and/or virtual storage locations). This may facilitate adramatic reduction in the amount of data storage required to handle theenormous amount of data produced by or about value chain networkentities 652 as applications 630 and uses of value chain networks growand proliferate. For example, a supply chain or inventory managementapplication in the value chain management platform 604, such as one forordering replacement parts for a machine or item of equipment, mayaccess the same data set about what parts have been replaced for a setof machines as a predictive maintenance application that is used topredict whether a component of a ship, or facility of a port is likelyto require replacement parts. Similarly, prediction may be used withrespect to the resupply of items.

In embodiments, value chain network data objects 1004 may be providedaccording to an object-oriented data model that defines classes,objects, attributes, parameters and other features of the set of dataobjects (such as associated with value chain network entities 652 andapplications 630) that are handled by the platform 604.

In embodiments, the data storage systems layer 624 may provide anextremely rich environment for collection of data that can be used forextraction of features or inputs for intelligence systems, such asexpert systems, analytic systems, artificial intelligence systems,robotic process automation systems, machine learning systems, deeplearning systems, supervised learning systems, or other intelligentsystems as disclosed throughout this disclosure and the documentsincorporated herein by reference. As a result, each application 630 inthe platform 604 and each adaptive intelligent system in the adaptiveintelligent systems layer 614 can benefit from the data collected orproduced by or for each of the others. In embodiments, the data storagesystems layer 624 may facilitate collection of data that can be used forextraction of features or inputs for intelligence systems such as adevelopment framework from artificial intelligence. In examples, thecollections of data may pull in and/or house event logs (naturallystored or ad-hoc, as needed), perform periodic checks on onboarddiagnostic data, or the like. In examples, pre calculation of featuresmay be deployed using AWS Lambda, for example, or various othercloud-based on-demand compute capabilities, such as pre-calculations,multiplexing signals. In many examples, there are pairings (doubles,triples, quadruplets, etc.) of similar kinds of value chain entitiesthat may use one or more sets of capabilities of the data handlinglayers 608 to deploy connectivity and services across value chainentities and across applications used by the entities even when amassinghundreds and hundreds of data types from relatively disparate entities.In these examples, various pairings of similar types of value chainentities using, at least in part, the connectivity and services acrossvalue chain entities and applications, may direct the information fromthe pairings of connected data to artificial intelligence servicesincluding the various neural networks disclosed herein and hybridcombinations thereof. In these examples, genetic programming techniquesmay be deployed to prune some of the input features in the informationfrom the pairings of connected data. In these examples, geneticprogramming techniques may also be deployed to add to and augment theinput features in the information from the pairings. These geneticprogramming techniques may be shown to increase the efficacy of thedeterminations established by the artificial intelligence services. Inthese examples, the information from the pairings of connected data maybe migrated to other layers on the platform including to support ordeploy robotic process automation, prediction, forecasting, and otherresources such that the shared data schema may facilitate ascapabilities and resources for the platform 604.

A wide range of data types may be stored in the storage layer 624 usingvarious storage media and data storage types, data architectures 1002,and formats, including, without limitation: asset and facility data1030, state data 1140 (such as indicating a state, condition status, orother indicator with respect to any of the value chain network entities652, any of the applications 630 or components or workflows thereof, orany of the components or elements of the platform 604, among others),worker data 1032 (including identity data, role data, task data,workflow data, health data, attention data, mood data, stress data,physiological data, performance data, quality data and many othertypes); event data 1034 ((such as with respect to any of a wide range ofevents, including operational data, transactional data, workflow data,maintenance data, and many other types of data that includes or relatesto events that occur within a value chain network 668 or with respect toone or more applications 630, including process events, financialevents, transaction events, output events, input events, state-changeevents, operating events, workflow events, repair events, maintenanceevents, service events, damage events, injury events, replacementevents, refueling events, recharging events, shipping events,warehousing events, transfers of goods, crossing of borders, moving ofcargo, inspection events, supply events, and many others); claims data664 (such as relating to insurance claims, such as for businessinterruption insurance, product liability insurance, insurance on goods,facilities, or equipment, flood insurance, insurance forcontract-related risks, and many others, as well as claims data relatingto product liability, general liability, workers compensation, injuryand other liability claims and claims data relating to contracts, suchas supply contract performance claims, product delivery requirements,warranty claims, indemnification claims, delivery requirements, timingrequirements, milestones, key performance indicators and others);accounting data 730 (such as data relating to completion of contractrequirements, satisfaction of bonds, payment of duties and tariffs, andothers); and risk management data 732 (such as relating to itemssupplied, amounts, pricing, delivery, sources, routes, customsinformation and many others), among many other data types associatedwith value chain network entities 652 and applications 630.

In embodiments, the data handling layers 608 are configured in atopology that facilitates shared adaptation capabilities, which may beprovided, managed, mediated and the like by one or more of a set ofservices, components, programs, systems, or capabilities of the adaptiveintelligent systems layer 614, referred to in some cases herein forconvenience as the adaptive intelligence layer 614. The adaptiveintelligence systems layer 614 may include a set of data processing,artificial intelligence and computational systems 634 that are describedin more detail elsewhere throughout this disclosure. Thus, use ofvarious resources, such as computing resources (such as availableprocessing cores, available servers, available edge computing resources,available on-device resources (for single devices or peered networks),and available cloud infrastructure, among others), data storageresources (including local storage on devices, storage resources in oron value chain entities or environments (including on-device storage,storage on asset tags, local area network storage and the like), networkstorage resources, cloud-based storage resources, database resources andothers), networking resources (including cellular network spectrum,wireless network resources, fixed network resources and others), energyresources (such as available battery power, available renewable energy,fuel, grid-based power, and many others) and others may be optimized ina coordinated or shared way on behalf of an operator, enterprise, or thelike, such as for the benefit of multiple applications, programs,workflows, or the like. For example, the adaptive intelligence layer 614may manage and provision available network resources for both a supplychain management application and for a demand planning application(among many other possibilities), such that low latency resources areused for supply chain management application (where rapid decisions maybe important) and longer latency resources are used for the demandplanning application. As described in more detail throughout thisdisclosure and the documents incorporated herein by reference, a widevariety of adaptations may be provided on behalf of the various servicesand capabilities across the various layers 624, including ones based onapplication requirements, quality of service, on-time delivery, serviceobjectives, budgets, costs, pricing, risk factors, operationalobjectives, efficiency objectives, optimization parameters, returns oninvestment, profitability, uptime/downtime, worker utilization, and manyothers.

The value chain management platform 604, referred to in some casesherein for convenience as the platform 604, may include, integrate with,and enable the various value chain network processes, workflows,activities, events and applications 630 described throughout thisdisclosure that enable an operator to manage more than one aspect of avalue chain network environment or entity 652 in a common applicationenvironment (e.g., shared, pooled, similarly licenses whether shareddata for one person, multiple people, or anonymized), such as one thattakes advantage of common data storage in the data storage layer 624,common data collection or monitoring in the monitoring systems layer 614and/or common adaptive intelligence of the adaptive intelligence layer614. Outputs from the applications 630 in the platform 604 may beprovided to the other data handing layers 624. These may include,without limitation, state and status information for various objects,entities, processes, flows and the like; object information, such asidentity, attribute and parameter information for various classes ofobjects of various data types; event and change information, such as forworkflows, dynamic systems, processes, procedures, protocols,algorithms, and other flows, including timing information; outcomeinformation, such as indications of success and failure, indications ofprocess or milestone completion, indications of correct or incorrectpredictions, indications of correct or incorrect labeling orclassification, and success metrics (including relating to yield,engagement, return on investment, profitability, efficiency, timeliness,quality of service, quality of product, customer satisfaction, andothers) among others. Outputs from each application 630 can be stored inthe data storage layer 624, distributed for processing by the datacollection layer 614, and used by the adaptive intelligence layer 614.The cross-application nature of the platform 604 thus facilitatesconvenient organization of all of the necessary infrastructure elementsfor adding intelligence to any given application, such as by supplyingmachine learning on outcomes across applications, providing enrichmentof automation of a given application via machine learning based onoutcomes from other applications or other elements of the platform 604,and allowing application developers to focus on application-nativeprocesses while benefiting from other capabilities of the platform 604.In examples, there may be systems, components, services and othercapabilities that optimize control, automation, or one or moreperformance characteristics of one or more value chain network entities652; or ones that may generally improve any of process and applicationoutputs and outcomes 1040 pursued by use of the platform 604. In someexamples, outputs and outcomes 1040 from various applications 630 may beused to facilitate automated learning and improvement of classification,prediction, or the like that is involved in a step of a process that isintended to be automated.

Some Data Storage Layer Details—Alternative Data Architectures

Referring to FIG. 12 , additional details, components, sub-systems, andother elements of an optional embodiment of the data storage layer 624of the platform 604 are illustrated. Various data architectures may beused, including conventional relational and object-oriented dataarchitectures, blockchain architectures 1180, asset tag data storagearchitectures 1178, local storage architectures 1190, network storagearchitectures 1174, multi-tenant architectures 1132, distributed dataarchitectures 1002, value chain network (VCN) data object architectures1004, cluster-based architectures 1128, event data-based architectures1034, state data-based architectures 1140, graph database architectures1124, self-organizing architectures 1134, and other data architectures1002.

The adaptive intelligent systems layer 614 of the platform 604 mayinclude one or more protocol adaptors 1110 for facilitating datastorage, retrieval access, query management, loading, extraction,normalization, and/or transformation to enable use of the various otherdata storage architectures 1002, such as allowing extraction from oneform of database and loading to a data system that uses a differentprotocol or data structure.

In embodiments, the value chain network-oriented data storage systemslayer 624 may include, without limitation, physical storage systems,virtual storage systems, local storage systems (e.g., part of the localstorage architectures 1190), distributed storage systems, databases,memory, network-based storage, network-attached storage systems (e.g.,part of the network storage architectures 1174 such as using NVME,storage attached networks, and other network storage systems), and manyothers.

In embodiments, the storage layer 624 may store data in one or moreknowledge graphs (such as a directed acyclic graph, a data map, a datahierarchy, a data cluster including links and nodes, a self-organizingmap, or the like) in the graph database architectures 1124. In exampleembodiments, the knowledge graph may be a prevalent example of when agraph database and graph database architecture may be used. In someexamples, the knowledge graph may be used to graph a workflow. For alinear workflow, a directed acyclic graph may be used. For a contingentworkflow, a cyclic graph may be used. The graph database (e.g., graphdatabase architectures 1124) may include the knowledge graph or theknowledge graph may be an example of the graph database. In exampleembodiments, the knowledge graph may include ontology and connections(e.g., relationships) between the ontology of the knowledge graph. In anexample, the knowledge graph may be used to capture an articulation ofknowledge domains of a human expert such that there may be anidentification of opportunities to design and build robotic processautomation or other intelligence that may replicate this knowledge set.The platform may be used to recognize that a type of expert is usingthis factual knowledge base (from the knowledge graph) coupled withcompetencies that may be replicable by artificial intelligence that maybe different depending on type of expertise involved. For example,artificial intelligence such as a convolutional neural network may beused with spatiotemporal aspects that may be used to diagnose issues orpacking up a box in a warehouse. Whereas the platform may use adifferent type of knowledge graph for a self-organizing map of an expertwhose main job is to segment customers into customer segmentationgroups. In some examples, the knowledge graph may be built from variousdata such as job credentials, job listings, parsing output deliverables.In embodiments, the data storage layer 624 may store data in a digitalthread, ledger, or the like, such as for maintaining a serial or otherrecords of an entities 652 over time, including any of the entitiesdescribed herein. In embodiments, the data storage layer 624 may use andenable an asset tag 1178, which may include a data structure that isassociated with an asset and accessible and managed, such as by use ofaccess controls, so that storage and retrieval of data is optionallylinked to local processes, but also optionally open to remote retrievaland storage options. In embodiments, the storage layer 624 may includeone or more blockchains 1180, such as ones that store identity data,transaction data, historical interaction data, and the like, such aswith access control that may be role-based or may be based oncredentials associated with a value chain entity 652, a service, or oneor more applications 630. Data stored by the data storage systems 624may include accounting and other financial data 730, access data 734,asset and facility data 1030 (such as for any of the value chain assetsand facilities described herein), asset tag data 1178, worker data 1032,event data 1034, risk management data 732, pricing data 738, safety data664 and many other types of data that may be associated with, producedby, or produced about any of the value chain entities and activitiesdescribed herein and in the documents incorporated by reference.

Adaptive Intelligent Systems and Monitoring Layers

Referring to FIG. 13 , additional details, components, sub-systems, andother elements of an optional embodiment of the platform 604 areillustrated. The management platform 604 may, in various optionalembodiments, include the set of applications 614, by which an operatoror owner of a value chain network entity, or other users, may manage,monitor, control, analyze, or otherwise interact with one or moreelements of a value chain network entity 652, such as any of theelements noted in connection above and throughout this disclosure.

In embodiments, the adaptive intelligent systems layer 614 may include aset of systems, components, services and other capabilities thatcollectively facilitate the coordinated development and deployment ofintelligent systems, such as ones that can enhance one or more of theapplications 630 at the application platform 604; ones that can improvethe performance of one or more of the components, or the overallperformance (e.g., speed/latency, reliability, quality of service, costreduction, or other factors) of the connectivity facilities 642; onesthat can improve other capabilities within the adaptive intelligentsystems layer 614; ones that improve the performance (e.g.,speed/latency, energy utilization, storage capacity, storage efficiency,reliability, security, or the like) of one or more of the components, orthe overall performance, of the value chain network-oriented datastorage systems 624; ones that optimize control, automation, or one ormore performance characteristics of one or more value chain networkentities 652; or ones that generally improve any of the process andapplication outputs and outcomes 1040 pursued by use of the platform604.

These adaptive intelligent systems 614 may include a robotic processautomation system 1442, a set of protocol adaptors 1110, a packetacceleration system 1410, an edge intelligence system 1420 (which may bea self-adaptive system), an adaptive networking system 1430, a set ofstate and event managers 1450, a set of opportunity miners 1460, a setof artificial intelligence systems 1160, a set of digital twin systems1700, a set of entity interaction systems 1920 (such as for setting up,provisioning, configuring and otherwise managing sets of interactionsbetween and among sets of value chain network entities 652 in the valuechain network 668), and other systems.

In embodiments, the value chain monitoring systems layer 614 and itsdata collection systems 640 may include a wide range of systems for thecollection of data. This layer may include, without limitation, realtime monitoring systems 1520 (such as onboard monitoring systems likeevent and status reporting systems on ships and other floating assets,on delivery vehicles, on trucks and other hauling assets, and inshipyards, ports, warehouses, distribution centers and other locations;on-board diagnostic (OBD) and telematics systems on floating assets,vehicles and equipment; systems providing diagnostic codes and eventsvia an event bus, communication port, or other communication system;monitoring infrastructure (such as cameras, motion sensors, beacons,RFID systems, smart lighting systems, asset tracking systems, persontracking systems, and ambient sensing systems located in variousenvironments where value chain activities and other events take place),as well as removable and replaceable monitoring systems, such asportable and mobile data collectors, RFID and other tag readers, smartphones, tablets and other mobile devices that are capable of datacollection and the like); software interaction observation systems 1500(such as for logging and tracking events involved in interactions ofusers with software user interfaces, such as mouse movements, touchpadinteractions, mouse clicks, cursor movements, keyboard interactions,navigation actions, eye movements, finger movements, gestures, menuselections, and many others, as well as software interactions that occuras a result of other programs, such as over APIs, among many others);mobile data collectors 1170 (such as described extensively herein and indocuments incorporated by reference), visual monitoring systems 1930(such as using video and still imaging systems, LIDAR, IR and othersystems that allow visualization of items, people, materials,components, machines, equipment, personnel, gestures, expressions,positions, locations, configurations, and other factors or parameters ofentities 652, as well as inspection systems that monitor processes,activities of workers and the like); point of interaction systems 1530(such as dashboards, user interfaces, and control systems for valuechain entities); physical process observation systems 1510 (such as fortracking physical activities of operators, workers, customers, or thelike, physical activities of individuals (such as shippers, deliveryworkers, packers, pickers, assembly personnel, customers, merchants,vendors, distributors and others), physical interactions of workers withother workers, interactions of workers with physical entities likemachines and equipment, and interactions of physical entities with otherphysical entities, including, without limitation, by use of video andstill image cameras, motion sensing systems (such as including opticalsensors, LIDAR, IR and other sensor sets), robotic motion trackingsystems (such as tracking movements of systems attached to a human or aphysical entity) and many others; machine state monitoring systems 1940(including onboard monitors and external monitors of conditions, states,operating parameters, or other measures of the condition of any valuechain entity, such as a machine or component thereof, such as a machine,such as a client, a server, a cloud resource, a control system, adisplay screen, a sensor, a camera, a vehicle, a robot, or othermachine); sensors and cameras 1950 and other IoT data collection systems1172 (including onboard sensors, sensors or other data collectors(including click tracking sensors) in or about a value chain environment(such as, without limitation, a point of origin, a loading or unloadingdock, a vehicle or floating asset used to convey goods, a container, aport, a distribution center, a storage facility, a warehouse, a deliveryvehicle, and a point of destination), cameras for monitoring an entireenvironment, dedicated cameras for a particular machine, process,worker, or the like, wearable cameras, portable cameras, camerasdisposed on mobile robots, cameras of portable devices like smart phonesand tablets, and many others, including any of the many sensor typesdisclosed throughout this disclosure or in the documents incorporatedherein by reference); indoor location monitoring systems 1532 (includingcameras, IR systems, motion-detection systems, beacons, RFID readers,smart lighting systems, triangulation systems, RF and other spectrumdetection systems, time-of-flight systems, chemical noses and otherchemical sensor sets, as well as other sensors); user feedback systems1534 (including survey systems, touch pads, voice-based feedbacksystems, rating systems, expression monitoring systems, affectmonitoring systems, gesture monitoring systems, and others); behavioralmonitoring systems 1538 (such as for monitoring movements, shoppingbehavior, buying behavior, clicking behavior, behavior indicating fraudor deception, user interface interactions, product return behavior,behavior indicative of interest, attention, boredom or the like,mood-indicating behavior (such as fidgeting, staying still, movingcloser, or changing posture) and many others); and any of a wide varietyof Internet of Things (IoT) data collectors 1172, such as thosedescribed throughout this disclosure and in the documents incorporatedby reference herein.

In embodiments, the value chain monitoring systems layer 614 and itsdata collection systems 640 may include an entity discovery system 1900for discovering one or more value chain network entities 652, such asany of the entities described throughout this disclosure. This mayinclude components or sub-systems for searching for entities within thevalue chain network 668, such as by device identifier, by networklocation, by geolocation (such as by geofence), by indoor location (suchas by proximity to known resources, such as IoT-enabled devices andinfrastructure, Wifi routers, switches, or the like), by cellularlocation (such as by proximity to cellular towers), by identitymanagement systems (such as where an entity 652 is associated withanother entity 652, such as an owner, operator, user, or enterprise byan identifier that is assigned by and/or managed by the platform 604),and the like. Entity discovery 1900 may initiate a handshake among a setof devices, such as to initiate interactions that serve variousapplications 630 or other capabilities of the platform 604.

Referring to FIG. 14 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections. The management platformincludes a user interface 3020 that provides, among other things, a setof adaptive intelligence systems 614. The adaptive intelligence systems614 provide coordinated intelligence (including artificial intelligence1160, expert systems 3002, machine learning 3004, and the like) for aset of demand management applications 824 and for a set of supply chainapplications 812 for a category of goods 3010, which may be produced andsold through the value chain. The adaptive intelligence systems 614 maydeliver artificial intelligence 1160 through a set of data processing,artificial intelligence and computational systems 634. In embodiments,the adaptive intelligence systems 614 are selectable and/or configurablethrough the user interface 3020 so that one or more of the adaptiveintelligence systems 614 can operate on or in cooperation with the setsof value chain applications (e.g., demand management applications 824and supply chain applications 812). The adaptive intelligence systems614 may include artificial intelligence, including any of the variousexpert systems, artificial intelligence systems, neural networks,supervised learning systems, machine learning systems, deep learningsystems, and other systems described throughout this disclosure and inthe documents incorporated by reference.

In embodiments, user interface may include interfaces for configuring anartificial intelligence system 1160 to take inputs from selected datasources of the value chain (such as data sources used by the set ofdemand management applications 824 and/or the set of supply chainapplications 812) and supply them, such as to a neural network,artificial intelligence system 1160 or any of the other adaptiveintelligence systems 614 described throughout this disclosure and in thedocuments incorporated herein by reference to enhance, control, improve,optimize, configure, adapt or have another impact on a value chain forthe category of goods 3010. In embodiments, the selected data sources ofthe value chain may be applied either as inputs for classification orprediction, or as outcomes relating to the value chain, the category ofgoods 3010 and the like.

In embodiments, providing coordinated intelligence may include providingartificial intelligence capabilities, such as artificial intelligencesystems 1160 and the like. Artificial intelligence systems mayfacilitate coordinated intelligence for the set of demand managementapplications 824 or the set of supply chain applications 812 or both,such as for a category of goods, such as by processing data that isavailable in any of the data sources of the value chain, such as valuechain processes, bills of materials, manifests, delivery schedules,weather data, traffic data, goods design specifications, customercomplaint logs, customer reviews, Enterprise Resource Planning (ERP)System, Customer Relationship Management (CRM) System, CustomerExperience Management (CEM) System, Service Lifecycle Management (SLM)System, Product Lifecycle Management (PLM) System, and the like.

In embodiments, the user interface 3020 may provide access to, amongother things artificial intelligence capabilities, applications, systemsand the like for coordinating intelligence for applications of the valuechain and particularly for value chain applications for the category ofgoods 3010. The user interface 3020 may be adapted to receiveinformation descriptive of the category of goods 3010 and configure useraccess to the artificial intelligence capabilities responsive thereto,so that the user, through the user interface is guided to artificialintelligence capabilities that are suitable for use with value chainapplications (e.g., the set of demand management applications 824 andsupply chain applications 812) that contribute to goods/services in thecategory of goods 3010. The user interface 3020 may facilitate providingcoordinated intelligence that comprises artificial intelligencecapabilities that provide coordinated intelligence for a specificoperator and/or enterprise that participates in the supply chain for thecategory of goods.

In embodiments, the user interface 3020 may be configured to facilitatethe user selecting and/or configuring multiple artificial intelligencesystems 1160 for use with the value chain. The user interface maypresent the set of demand management applications 824 and supply chainapplications 812 as connected entities that receive, process, andproduce outputs each of which may be shared among the applications.Types of artificial intelligence systems 1160 may be indicated in theuser interface 3020 responsive to sets of connected applications ortheir data elements being indicated in the user interface, such as bythe user placing a pointer proximal to a connected set of applicationsand the like. In embodiments, the user interface 3020 may facilitateaccess to the set of adaptive intelligence systems provides a set ofcapabilities that facilitate development and deployment of intelligencefor at least one function selected from a list of functions consistingof supply chain application automation, demand management applicationautomation, machine learning, artificial intelligence, intelligenttransactions, intelligent operations, remote control, analytics,monitoring, reporting, state management, event management, and processmanagement.

The adaptive intelligence systems 614 may be configured with dataprocessing, artificial intelligence and computational systems 634 thatmay operate cooperatively to provide coordinated intelligence, such aswhen an artificial intelligence system 1160 operates on or responds todata collected by or produced by other systems of the adaptiveintelligence systems 614, such as a data processing system and the like.In embodiments, providing coordinated intelligence may include operatinga portion of a set of artificial intelligence systems 1160 that employsone or more types of neural network that is described herein and in thedocuments incorporated herein by reference and that processes any of thedemand management application outputs and supply chain applicationoutputs to provide the coordinated intelligence.

In embodiments, providing coordinated intelligence for the set of demandmanagement applications 824 may include configuring at least one of theadaptive intelligence systems 614 (e.g., through the user interface 3020and the like) for at least one or more demand management applicationsselected from a list of demand management applications including ademand planning application, a demand prediction application, a salesapplication, a future demand aggregation application, a marketingapplication, an advertising application, an e-commerce application, amarketing analytics application, a customer relationship managementapplication, a search engine optimization application, a salesmanagement application, an advertising network application, a behavioraltracking application, a marketing analytics application, alocation-based product or service-targeting application, a collaborativefiltering application, a recommendation engine for a product or service,and the like.

Similarly, providing coordinated intelligence for the set of supplychain applications 812 may include configuring at least one of theadaptive intelligence systems 614 for at least one or more supply chainapplications selected from a list of supply chain applications includinga goods timing management application, a goods quantity managementapplication, a logistics management application, a shipping application,a delivery application, an order for goods management application, anorder for components management application, and the like.

In embodiments, the management platform 102 may, such as through theuser interface 3020 facilitate access to the set of adaptiveintelligence systems 614 that provide coordinated intelligence for a setof demand management applications 824 and supply chain applications 812through the application of artificial intelligence. In such embodiments,the user may seek to align supply with demand while ensuringprofitability and the like of a value chain for a category of goods3010. By providing access to artificial intelligence capabilities 1160,the management platform allows the user to focus on the applications ofdemand and supply while gaining advantages of techniques such as expertsystems, artificial intelligence systems, neural networks, supervisedlearning systems, machine learning systems, deep learning systems, andthe like.

In embodiments, the management platform 102 may, through the userinterface 3020 and the like provide a set of adaptive intelligencesystems 614 that provide coordinated artificial intelligence 1160 forthe sets of demand management applications 824 and supply chainapplications 812 for the category of goods 3020 by, for example,determining (automatically) relationships among demand management andsupply chain applications based on inputs used by the applications,results produced by the applications, and value chain outcomes. Theartificial intelligence 1160 may be coordinated by, for example, the setof data processing, artificial intelligence and computational systems634 available through the adaptive intelligence systems 614.

In embodiments, the management platform 102 may be configured with a setof artificial intelligence systems 1160 as part of a set of adaptiveintelligence systems 614 that provide the coordinated intelligence forthe sets of demand management applications 824 and supply chainapplications 812 for a category of goods 3010. The set of artificialintelligence systems 1160 may provide the coordinated intelligence sothat at least one supply chain application of the set of supply chainapplications 812 produces results that address at least one aspect ofsupply for at least one of the goods in the category of goods asdetermined by at least one demand management application of the set ofdemand management applications 824. In examples, a behavioral trackingdemand management application may generate results for behavior of usesof a good in the category of goods 3010. The artificial intelligencesystems 1160 may process the behavior data and conclude that there is aperceived need for greater consumer access to a second product in thecategory of goods 3010. This coordinated intelligence may be, optionallyautomatically, applied to the set of supply chain applications 812 sothat, for example, production resources or other resources in the valuechain for the category of goods are allocated to the second product. Inexamples, a distributor who handles stocking retailer shelves mayreceive a new stocking plan that allocates more retail shelf space forthe second product, such as by taking away space from a lower marginproduct and the like.

In embodiments, the set of artificial intelligence systems 1160 and thelike may provide coordinated intelligence for the sets of supply chainand demand management applications by, for example, determining anoptionally temporal prioritization of demand management applicationoutputs that impact control of supply chain applications so that anoptionally temporal demand for at least one of the goods in the categoryof goods 3010 can be met. Seasonal adjustments in prioritization ofdemand application results are one example of a temporal change.Adjustments in prioritization may also be localized, such as when alarge college football team is playing at their home stadium and localsupply of tailgating supplies may temporally be adjusted even thoughdemand management application results suggest that small propane stovesare not currently in demand in a wider region.

A set of adaptive intelligence systems 614 that provide coordinatedintelligence, such as by providing artificial intelligence capabilities1160 and the like may also facilitate development and deployment ofintelligence for at least one function selected from a list of functionsconsisting of supply chain application automation, demand managementapplication automation, machine learning, artificial intelligence,intelligent transactions, intelligent operations, remote control,analytics, monitoring, reporting, state management, event management,and process management. The set of adaptive intelligence systems 614 maybe configured as a layer in the platform and an artificial intelligencesystem therein may operate on or be responsive to data collected byand/or produced by other systems (e.g., data processing systems, expertsystems, machine learning systems and the like) of the adaptiveintelligence systems layer.

In addition to providing coordinated intelligence configured forspecific categories of goods, the coordinated intelligence may beprovided for a specific value chain entity 652, such as a supply chainoperator, business, enterprise, and the like that participates in thesupply chain for the category of goods.

Providing coordinated intelligence may include employing a neuralnetwork to process at least one of the inputs and outputs of the sets ofdemand management and supply chain applications. Neural networks may beused with demand applications, such as a demand planning application, ademand prediction application, a sales application, a future demandaggregation application, a marketing application, an advertisingapplication, an e-commerce application, a marketing analyticsapplication, a customer relationship management application, a searchengine optimization application, a sales management application, anadvertising network application, a behavioral tracking application, amarketing analytics application, a location-based product orservice-targeting application, a collaborative filtering application, arecommendation engine for a product or service, and the like. Neuralnetworks may also be used with supply chain applications such as a goodstiming management application, a goods quantity management application,a logistics management application, a shipping application, a deliveryapplication, an order for goods management application, an order forcomponents management application, and the like. Neural networks mayprovide coordinated intelligence by processing data that is available inany of a plurality of value chain data sources for the category of goodsincluding without limitation processes, bill of materials, weather,traffic, design specification, customer complaint logs, customerreviews, Enterprise Resource Planning (ERP) System, CustomerRelationship Management (CRM) System, Customer Experience Management(CEM) System, Service Lifecycle Management (SLM) System, ProductLifecycle Management (PLM) System, and the like. Neural networksconfigured for providing coordinated intelligence may share adaptationcapabilities with other adaptive intelligence systems 614, such as whenthese systems are configured in a topology that facilitates such sharedadaptation. In embodiments, neural networks may facilitate provisioningavailable value chain/supply chain network resources for both the set ofdemand management applications and for the set of supply chainapplications. In embodiments, neural networks may provide coordinatedintelligence to improve at least one of the list of outputs consistingof a process output, an application output, a process outcome, anapplication outcome, and the like.

Referring to FIG. 15 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections. The management platformincludes a user interface 3020 that provides, among other things, ahybrid set of adaptive intelligence systems 614. The hybrid set ofadaptive intelligence systems 614 provide coordinated intelligencethrough the application of artificial intelligence, such as throughapplication of a hybrid artificial intelligence system 3060, andoptionally through one or more expert systems, machine learning systems,and the like for use with a set of demand management applications 824and for a set of supply chain applications 812 for a category of goods3010, which may be produced and sold through the value chain. The hybridadaptive intelligence systems 614 may deliver two types of artificialintelligence systems, type A 3052 and type B 3054 through a set of dataprocessing, artificial intelligence and computational systems 634. Inembodiments, the hybrid adaptive intelligence systems 614 are selectableand/or configurable through the user interface 3020 so that one or moreof the hybrid adaptive intelligence systems 614 can operate on or incooperation with the sets of supply chain applications (e.g., demandmanagement applications 824 and supply chain applications 812). Thehybrid adaptive intelligence systems 614 may include a hybrid artificialintelligence system 3060 that may include at least two types ofartificial intelligence capabilities including any of the various expertsystems, artificial intelligence systems, neural networks, supervisedlearning systems, machine learning systems, deep learning systems, andother systems described throughout this disclosure and in the documentsincorporated by reference. The hybrid adaptive intelligence systems 614may facilitate applying a first type of artificial intelligence system1160 to the set of demand management applications 824 and a second typeof artificial intelligence system 1160 to the set of supply chainapplications 812, wherein each of the first type and second type ofartificial intelligence system 1160 can operate independently,cooperatively, and optionally coordinate operation to providecoordinated intelligence for operation of the value chain that producesat least one of the goods in the category of goods 3010.

In embodiments, the user interface 3020 may include interfaces forconfiguring a hybrid artificial intelligence system 3060 to take inputsfrom selected data sources of the value chain (such as data sources usedby the set of demand management applications 824 and/or the set ofsupply chain applications 812) and supply them, such as to at least oneof the two types of artificial intelligence systems in the hybridartificial intelligence system 3060, types of which are describedthroughout this disclosure and in the documents incorporated herein byreference to enhance, control, improve, optimize, configure, adapt orhave another impact on a value chain for the category of goods 3010. Inembodiments, the selected data sources of the value chain may be appliedeither as inputs for classification or prediction, or as outcomesrelating to the value chain, the category of goods 3010 and the like.

In embodiments, the hybrid adaptive intelligence systems 614 provides aplurality of distinct artificial intelligence systems 1160, a hybridartificial intelligence system 3060, and combinations thereof. Inembodiments, any of the plurality of distinct artificial intelligencesystems 1160 and the hybrid artificial intelligence system 3060 may beconfigured as a plurality of neural network-based systems, such as aclassification-adapted neural network, a prediction-adapted neuralnetwork and the like. As an example of hybrid adaptive intelligencesystems 614, a machine learning-based artificial intelligence system maybe provided for the set of demand management applications 824 and aneural network-based artificial intelligence system may be provided forthe set of supply chain applications 812. As an example of a hybridartificial intelligence system 3060, the hybrid adaptive intelligencesystems 614 may provide the hybrid artificial intelligence system 3060that may include a first type of artificial intelligence that is appliedto the demand management applications 824 and which is distinct from asecond type of artificial intelligence that is applied to the supplychain applications 812. A hybrid artificial intelligence system 3060 mayinclude any combination of types of artificial intelligence systemsincluding a plurality of a first type of artificial intelligence (e.g.,neural networks) and at least one second type of artificial intelligence(e.g., an expert system) and the like. In embodiments, a hybridartificial intelligence system may comprise a hybrid neural network thatapplies a first type of neural network with respect to the demandmanagement applications 824 and a second type of neural network withrespect to the supply chain applications 812. Yet further, a hybridartificial intelligence system 3060 may provide two types of artificialintelligence to different applications, such as different demandmanagement applications 824 (e.g., a sales management application and ademand prediction application) or different supply chain applications812 (e.g., a logistics control application and a production qualitycontrol application).

In embodiments, hybrid adaptive intelligence systems 614 may be appliedas distinct artificial intelligence capabilities to distinct demandmanagement applications 824. As examples, coordinated intelligencethrough a hybrid artificial intelligence capabilities may be provided toa demand planning application by a feed-forward neural network, to ademand prediction application by a machine learning system, to a salesapplication by a self-organizing neural network, to a future demandaggregation application by a radial basis function neural network, to amarketing application by a convolutional neural network, to anadvertising application by a recurrent neural network, to an e-commerceapplication by a hierarchical neural network, to a marketing analyticsapplication by a stochastic neural network, to a customer relationshipmanagement application by an associative neural network and the like.

Referring to FIG. 16 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections for providing a set ofpredictions 3070. The management platform includes a user interface 3020that provides, among other things, a set of adaptive intelligencesystems 614. The adaptive intelligence systems 614 provide a set ofpredictions 3070 through the application of artificial intelligence,such as through application of an artificial intelligence system 1160,and optionally through one or more expert systems, machine learningsystems, and the like for use with a coordinated set of demandmanagement applications 824 and supply chain applications 812 for acategory of goods 3010, which may be produced and sold through the valuechain. The adaptive intelligence systems 614 may deliver the set ofprediction 3070 through a set of data processing, artificialintelligence and computational systems 634. In embodiments, the adaptiveintelligence systems 614 are selectable and/or configurable through theuser interface 3020 so that one or more of the adaptive intelligencesystems 614 can operate on or in cooperation with the coordinated setsof value chain applications. The adaptive intelligence systems 614 mayinclude an artificial intelligence system that provides artificialintelligence capabilities known to be associated with artificialintelligence including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporatedby reference. The adaptive intelligence systems 614 may facilitateapplying adapted intelligence capabilities to the coordinated set ofdemand management applications 824 and supply chain applications 812such as by producing a set of predictions 3070 that may facilitatecoordinating the two sets of value chain applications, or at leastfacilitate coordinating at least one demand management application andat least one supply chain application from their respective sets.

In embodiments, the set of predictions 3070 includes a least oneprediction of an impact on a supply chain application based on a currentstate of a coordinated demand management application, such as aprediction that a demand for a good will decrease earlier thanpreviously anticipated. The converse may also be true in that the set ofpredictions 3070 includes at least one prediction of an impact on ademand management application based on a current state of a coordinatedsupply chain application, such as a prediction that a lack of supply ofa good will likely impact a measure of demand of related goods. Inembodiments, the set of predictions 3070 is a set of predictions ofadjustments in supply required to meet demand. Other predictions includeat least one prediction of change in demand that impacts supply. Yetother predictions in the set of predictions predict a change in supplythat impacts at least one of the set of demand management applications,such as a promotion application for at least one good in the category ofgoods. A prediction in the set of predictions may be as simple assetting a likelihood that a supply of a good in the category of goodswill not meet demand set by a demand setting application.

In embodiments, the adaptive intelligence systems 614 may provide a setof artificial intelligence capabilities to facilitate providing the setof predictions for the coordinated set of demand management applicationsand supply chain applications. In one non-limiting example, the set ofartificial intelligence capabilities may include a probabilistic neuralnetwork that may be used to predict a fault condition or a problem stateof a demand management application such as a lack of sufficientvalidated feedback. The probabilistic neural network may be used topredict a problem state with a machine performing a value chainoperation (e.g., a production machine, an automated handling machine, apackaging machine, a shipping machine and the like) based on acollection of machine operating information and preventive maintenanceinformation for the machine.

In embodiments, the set of predictions 3070 may be provided by themanagement platform 102 directly through a set of adaptive artificialintelligence systems.

In embodiments, the set of predictions 3070 may be provided for thecoordinated set of demand management applications and supply chainapplications for a category of goods by applying artificial intelligencecapabilities for coordinating the set of demand management applicationsand supply chain applications.

In embodiments, the set of predictions 3070 may be predictions ofoutcomes for operating a value chain with the coordinated set demandmanagement applications and supply chain applications for the categoryof goods, so that a user may conduct test cases of coordinated sets ofdemand management applications and supply chain applications todetermine which sets may produce desirable outcomes (viable candidatesfor a coordinated set of applications) and which may produce undesirableoutcomes.

Referring to FIG. 17 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections for providing a set ofclassifications 3080. The management platform includes a user interface3020 that provides, among other things, a set of adaptive intelligencesystems 614. The adaptive intelligence systems 614 provide a set ofclassifications 3080 through, for example, the application of artificialintelligence, such as through application of an artificial intelligencesystem 1160, and optionally through one or more expert systems, machinelearning systems, and the like for use with a coordinated set of demandmanagement applications 824 and supply chain applications 812 for acategory of goods 3010, which may be produced, marketed, sold, resold,rented, leased, given away, serviced, recycled, renewed, enhanced, andthe like through the value chain. The adaptive intelligence systems 614may deliver the set of classifications 3080 through a set of dataprocessing, artificial intelligence and computational systems 634. Inembodiments, the adaptive intelligence systems 614 are selectable and/orconfigurable through the user interface 3020 so that one or more of theadaptive intelligence systems 614 can operate on or in cooperation withthe coordinated sets of value chain applications. The adaptiveintelligence systems 614 may include an artificial intelligence systemthat provides, among other things classification capabilities throughany of the various expert systems, artificial intelligence systems,neural networks, supervised learning systems, machine learning systems,deep learning systems, and other systems described throughout thisdisclosure and in the documents incorporated by reference. The adaptiveintelligence systems 614 may facilitate applying adapted intelligencecapabilities to the coordinated set of demand management applications824 and supply chain applications 812 such as by producing a set ofclassifications 3080 that may facilitate coordinating the two sets ofvalue chain applications, or at least facilitate coordinating at leastone demand management application and at least one supply chainapplication from their respective sets.

In embodiments, the set of classifications 3080 includes at least oneclassification of a current state of a supply chain application for useby a coordinated demand management application, such as a classificationof a problem state that may impact operation of a demand managementapplication, such as a marketing application and the like. Such aclassification may be useful in determining how to adjust a marketexpectation for a good that is going to have a lower yield thanpreviously anticipated. The converse may also be true in that the set ofclassifications 3080 includes at least one classification of a currentstate of a demand management application and its relationship to acoordinated supply chain application. In embodiments, the set ofclassifications 3080 is a set of classifications of adjustments insupply required to meet demand, such as adjustments to production workerneeds would be classified differently that adjustments in third-partylogistics providers. Other classifications may include at least oneclassification of perceived changes in demand and a resulting potentialimpact on supply management. Yet other classifications in the set ofclassifications may include a supply chain application impact on atleast one of the set of demand management applications, such as apromotion application for at least one good in the category of goods. Aclassification in the set of classifications may be as simple asclassifying a likelihood that a supply of a good in the category ofgoods will not meet demand set by a demand setting application.

In embodiments, the adaptive intelligence systems 614 may provide a setof artificial intelligence capabilities to facilitate providing the setof classifications 3080 for the coordinated set of demand managementapplications and supply chain applications. In one non-limiting example,the set of artificial intelligence capabilities may include aprobabilistic neural network that may be used to classify faultconditions or problem states of a demand management application, such asa classification of a lack of sufficient validated feedback. Theprobabilistic neural network may be used to classify a problem state ofa machine performing a value chain operation (e.g., a productionmachine, an automated handling machine, a packaging machine, a shippingmachine and the like) as pertaining to at least one of machine operatinginformation and preventive maintenance information for the machine.

In embodiments, the set of classifications 3080 may be provided by themanagement platform 102 directly through a set of adaptive artificialintelligence systems. Further, the set of classifications 3080 may beprovided for the coordinated set of demand management applications andsupply chain applications for a category of goods by applying artificialintelligence capabilities for coordinating the set of demand managementapplications and supply chain applications.

In embodiments, the set of classifications 3080 may be classificationsof outcomes for operating a value chain with the coordinated set demandmanagement applications and supply chain applications for the categoryof goods, so that a user may conduct test cases of coordinated sets ofdemand management applications and supply chain applications todetermine which sets may produce outcomes that are classified asdesirable (e.g., viable candidates for a coordinated set ofapplications) and outcomes that are classified as undesirable.

In embodiments, the set of classifications may comprise a set ofadaptive intelligence functions, such as a neural network that may beadapted to classify information associated with the category of goods.In an example, the neural network may be a multilayered feed forwardneural network.

In embodiments, performing classifications may include classifyingdiscovered value chain entities as one of demand centric and supplycentric.

In embodiments, the set of classifications 3080 may be achieved throughuse of artificial intelligence systems 1160 for coordinating the set ofcoordinated demand management and supply chain applications. Artificialintelligence systems may configure and generate sets of classifications3080 as a means by which demand management applications and supply chainapplications can be coordinated. In an example, classification ofinformation flow throughout a value chain may be classified as beingrelevant to both a demand management application and a supply chainapplication; this common relevance may be a point of coordination amongthe applications. In embodiments, the set of classifications may beartificial intelligence generated classifications of outcomes ofoperating a supply chain that is dependent on the coordinated demandmanagement applications 824 and supply chain applications 812.

Referring to FIG. 18 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections for achieving automatedcontrol intelligence. The management platform includes a user interface3020 that provides, among other things, a set of adaptive intelligencesystems 614. The adaptive intelligence systems 614 provide automatedcontrol signaling 3092 for a coordinated set of demand managementapplications 824 and supply chain applications 812 for a category ofgoods 3010, which may be produced and sold through the value chain. Theadaptive intelligence systems 614 may deliver the automated controlsignals 3092 through a set of data processing, artificial intelligenceand computational systems 634. In embodiments, the adaptive intelligencesystems 614 are selectable and/or configurable through the userinterface 3020 so that one or more of the adaptive intelligence systems614 can automatically control the sets of supply chain applications(e.g., demand management applications 824 and supply chain applications812). The adaptive intelligence systems 614 may include artificialintelligence including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporatedby reference.

In embodiments, the user interface 3020 may include interfaces forconfiguring an adaptive intelligence systems 614 to take inputs fromselected data sources of the value chain 3094 (such as data sources usedby the coordinated set of demand management applications 824 and/or theset of supply chain applications 812) and supply them, such as to aneural network, artificial intelligence system 1160 or any of the otheradaptive intelligence systems 614 described throughout this disclosureand in the documents incorporated herein by reference for producingautomated control signals 3092, such as to enhance, control, improve,optimize, configure, adapt or have another impact on a value chain forthe category of goods 3010. In embodiments, the selected data sources ofthe value chain may be used for determining aspects of the automatedcontrol signals, such as for temporal adjustments to control outcomesrelating to the value chain at least for the category of goods 3010 andthe like.

In an example, the set of automated control signals may include at leastone control signal for automating execution of a supply chainapplication, such as a production start, an automated material order, aninventory check, a billing application and the like in the coordinatedset of demand management applications and supply chain applications. Inyet another example of automated control signal generation, the set ofautomated control signals may include at least one control signal forautomating execution of a demand management application, such as aproduct recall application, an email distribution application and thelike in the coordinated set of demand management applications and supplychain applications. In yet other examples, the automate control signalsmay control timing of demand management applications based on goodssupply status.

In embodiments, the adaptive intelligence systems 614 may apply machinelearning to outcomes of supply to automatically adapt a set of demandmanagement application control signals. Similarly, the adaptiveintelligence systems 614 may apply machine learning to outcomes ofdemand management to automatically adapt a set of supply chainapplication control signals. The adaptive intelligence systems 614 mayprovide further processing for automated control signal generation, suchas by applying artificial intelligence to determine aspects of a valuechain that impact automated control of the coordinated set of demandmanagement applications and supply chain applications for a category ofgoods. The determined aspects could be used in the generation andoperation of automated control intelligence/signals, such as byfiltering out value chain information for aspects that do not impact thetargeted demand management and supply chain applications.

Automated control of, for example, supply chain applications may berestricted, such as by policy, operational limits, safety constraintsand the like. The set of adaptive intelligence systems may determine arange of supply chain application control values within which controlcan be automated. In embodiments, the range may be associated with asupply rate, a supply timing rate, a mix of goods in a category ofgoods, and the like.

Embodiments are described herein for using artificial intelligencesystems or capabilities to identify, configure and regulate automatedcontrol signals. Such embodiments may further include a closed loop offeedback from the coordinated set of demand management and supply chainapplications (e.g., state information, output information, outcomes andthe like) that is optionally processed with machine learning and used toadapt the automated control signals for at least one of the goods in thecategory of goods. An automated control signal may be adapted based on,for example, an indication of feedback from a supply chain applicationthat yield of a good suggests a production problem. In this example, theautomated control signal may impact production rate and the feedback maycause the signal to automatically self-adjust to a slower productionrate until the production problem is resolved.

Referring to FIG. 19 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections for providing informationrouting recommendations. The management platform includes a set of valuechain networks 3102 from which network data 3110 is collected from a setof information routing activities, the information including outcomes,parameters, routing activity information and the like. Within the set ofvalue chain networks 3102 is selected a select value chain network 3104for which at least one information routing recommendation 3130 isprovided. An artificial intelligence system 1160 may include a machinelearning system and may be trained using a training set derived from thenetwork data 3110 outcomes, parameters and routing activity informationfor the set of value chain networks 3102. The artificial intelligencesystem 1160 may further provide an information routing recommendation3130 based on a current status 3120 of the select value chain network3104. The artificial intelligence system may use machine learning totrain on information transaction types within the set of value chainnetworks 3102, thereby learning pertinent factors regarding differenttransaction types (e.g., real-time inventory updates, buyer creditchecks, engineering signoff, and the like) and contributing to theinformation routing recommendation accordingly. The artificialintelligence system may also use machine learning to train oninformation value for different types and/or classes of informationrouted in and throughout the set of value chain networks 3102.Information may be valued on a wide range of factors, including timingof information availability and timing of information consumption aswell as information content-based value, such as information withoutwhich a value chain network element (e.g., a production provider) cannotperform a desired action (e.g., starting volume production without awork order). Therefore information routing recommendations may be basedon training on transaction type, information value, and a combinationthereof. These are merely exemplary information routing recommendationtraining and recommendation basis factors and are presented here withoutlimitation on other elements for training and recommendation basis.

In embodiments, the artificial intelligence system 1160 may provide aninformation routing recommendation 3130 based on transaction type,transaction type and information type, network type and the like. Aninformation routing recommendation may be based on combinations offactors, such as information type and network type, such as when aninformation type (streaming) is not compatible with a network type(small transactions).

In embodiments, the artificial intelligence system 1160 may use machinelearning to develop an understanding of networks within the selectedvalue chain network 3104, such as network topology, network loading,network reliability, network latency and the like. This understandingmay be combined with, for example, detected or anticipated networkconditions to form an information routing recommendation. Aspects suchas existence of edge intelligence in a value chain network 3104 caninfluence one or more information routing recommendations. In anexample, a type of information may be incompatible with a network type;however the network may be configured with edge intelligence that can beleveraged by the artificial intelligence system 1160 to adapt the formof the information being routed so that it is compatible with a targetednetwork type. This is also an example of more general consideration forinformation routing recommendation—network resources (e.g., presence,availability, and capability), such as edge computing, server access,network-based storage resources and the like. Likewise, value chainnetwork entities may impact information routing recommendations. Inembodiments, an information routing recommendation may avoid routinginformation that is confidential to a first supplier in the value chainthrough network nodes controlled by competitors of the supplier. Inembodiments, an information routing recommendation may include routinginformation to a first node where it is partially consumed and partiallyprocessed for further routing, such as by splitting up the portionpartially processed for further routing into destination-specificinformation sets.

In embodiments, an artificial intelligence system 1160 may provide aninformation routing recommendation based on goals, such as goals of avalue chain network, goals of information routing, and the like.Goal-based information routing recommendations may include routinggoals, such as Quality of Service routing goals, routing reliabilitygoals (which may be measured based on a transmission failure rate andthe like). Other goals may include a measure of latency associated withone or more candidate routes. An information routing recommendation maybe based on the availability of information in a selected value chainnetwork, such as when information is available and when it needs to bedelivered. For information that is available well ahead of when it isneeded (e.g., a nightly production report that is available for routingat 2 AM is first needed by 7 AM), routing recommendations may includeusing resources that are lower cost, may involve short delays in routingand the like. For information that is available just before it is needed(e.g., a result of product testing is needed within a few hundredmilliseconds of when the test is finished to maintain a productionoperation rate, and the like).

An information routing recommendation may be formed by the artificialintelligence system 1160 based on information persistence factors, suchas how long information is available for immediate routing within thevalue chain network. An information routing recommendation that factorsinformation persistence may select network resources based onavailability, cost and the like during a time of informationpersistence.

Information value and an impact on information value may factor into aninformation routing recommendation. As an example, information that isvalid for a single shipment (e.g., a production run of a good) maysubstantively lose value once the shipment has been satisfactorilyreceived. In such an example, an information routing recommendation mayindicate routing the relevant information to all of the highest priorityconsumers of the information while it is still valid. Likewise, routingof information that is consumed by more than one value chain entity mayneed to be coordinated so that each value chain entity receives theinformation at a desired time/moment, such as during the same productionshift, at their start of day, which may be different if the entities arein different time zones, and the like.

In embodiments, information routing recommendations may be based on atopology of a value chain, based on location and availability of networkstorage resources, and the like.

In embodiments, one or more information routing recommendations may beadapted while the information is routed based on, for example, changesin network resource availability, network resource discovery, networkdynamic loading, priority of recommendations that are generated afterinformation for a first recommendation is in-route, and the like.

Referring to FIG. 20 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections for semi-sentient problemrecognitions of pain points in a value chain network. The managementplatform includes a set of value chain network entities 3152 from whichentity-related data 3160 is collected and includes outcomes, parameters,activity information and the like associated with the entities. Withinthe set of value chain network entities 3152 is selected a set of selectvalue chain network entities 3154 for which at least one pain pointproblem state 3172 is detected. An artificial intelligence system 1160may be training on a training set derived from the entity-related data3160 including training on outcomes associated with value chainentities, parameters associated with, for example, operation of thevalue chain, value chain activity information and the like. Theartificial intelligence system may further employ machine learning tofacilitate learning problem state factors 3180 that may characterizeproblem states input as training data. These factors 3180 may further beused by an instance of artificial intelligence 1160′ that operates oncomputing resources 3170 that are local to value chain network entitiesthat are experiencing the problem/result of a pain point. A goal of sucha configuration of artificial intelligence systems, data sets, and valuechain networks is to recognize a problem state in a portion of theselected value chain.

In embodiments, recognizing problem states may be based on varianceanalysis, such as variances that occur in value chain measures (e.g.,loading, latency, delivery time, cost, and the like), particularly in aspecific measure over time. Variances that exceed a variance threshold(e.g., an optionally dynamic range of results of a value chainoperation, such as production, shipping, clearing customs, and the like)may be indicative of a pain point.

In addition to detecting problem states, the platform 102, such asthrough the methods of semi-sentient problem recognition, predict a painpoint based at least in part on a correlation with a detected problemstate. The correlation may be derived from the value chain, such as ashipper cannot deliver international goods until they are processedthrough customs, or a sales forecast cannot be provided with a highdegree of confidence without high quality field data and the like. Inembodiments, a predicted pain point may be a point of value chainactivity further along a supply chain, an activity that occurs in arelated activity (e.g., tax planning is related to tax laws), and thelike. A predicted pain point may be assigned a risk value based onaspects of the detected problem state and correlations between thepredicted pain point activity and the problem state activity. If aproduction operation can receive materials from two suppliers, a problemstate with one of the suppliers may indicate a low risk of a pain pointof use of the material. Likewise, if a demand management applicationindicates high demand for a good and a problem is detected withinformation on which the demand is based, a risk of excess inventory(pain point) may be high depending on, for example how far along in thevalue chain the good has progressed.

In embodiments, semi-sentient problem recognition may involve more thanmere linkages of data and operational states of entities engaged in avalue chain. Problem recognition may also be based on human factors,such as perceived stress of production supervisors, shippers, and thelike. Human factors for use in semi-sentient problem recognition may becollected from sensors that facilitate detection of human stress leveland the like (e.g., wearable physiological sensors, and the like).

In embodiments, semi-sentient problem recognition may also be based onunstructured information, such as digital communication, voicemessaging, and the like that may be shared among, originate with, or bereceived by humans involved in the value chain operations. As anexample, natural language processing of email communications amongworkers in an enterprise may indicate a degree of discomfort with, forexample, a supplier to a value chain. While data associated with thesupplier (e.g., on-time production, quality, and the like) may be withina variance range deemed acceptable, information within this unstructuredcontent may indicate a potential pain point, such as a personal issuewith a key participant at the supplier and the like. By employingnatural language processing, artificial intelligence, and optionallymachine learning, problem state recognition may be enhanced.

In embodiments, semi-sentient problem recognition may be based onanalysis of variances of measures of a value chainoperation/entity/application including variance of a given measure overtime, variance of two related measures, and the like. In embodiments,variance in outcomes over time may indicate a problem state and/orsuggest a pain point. In embodiments, an artificial intelligence-basedsystem may determine an acceptable range of outcome variance and applythat range to measures of a select set of value chain network entities,such as entities that share one or more similarities, to facilitatedetection of a problem state. In embodiments, an acceptable range ofoutcome variance may indicate a problem state trigger threshold that maybe used by a local instance of artificial intelligence to signal aproblem state. In such a scenario, a problem state may be detected whenat least one measure of the value chain activity/entity and the like isgreater than the artificial intelligence-determined problem statethreshold. Variance analysis for problem state detection may includedetecting variances in start/end times of scheduled value chain networkentity activities, variances in at least one of production time,production quality, production rate, production start time, productionresource availability or trends thereof, variances in a measure ofshipping supply chain entity, variances in a duration of time fortransfer from one mode of transport to another (e.g., when the varianceis greater than a transport mode problem state threshold), variances inquality testing, and the like.

In embodiments, a semi-sentient problem recognition system may include amachine learning/artificial intelligence prediction of a correlated painpoint further along a supply chain due to a detected pain point, such asa risk and/or need for overtime, expedited shipping, discounting goodsprices, and the like.

In embodiments, a machine learning/artificial intelligence system mayprocess outcomes, parameters, and data collected from a set of datasources relating to a set of value chain entities and activities todetect at least one pain point selected from the list of pain pointsconsisting of late shipment, damaged container, damaged goods, wronggoods, customs delay, unpaid duties, weather event, damagedinfrastructure, blocked waterway, incompatible infrastructure, congestedport, congested handling infrastructure, congested roadway, congesteddistribution center, rejected goods, returned goods, waste material,wasted energy, wasted labor force, untrained workforce, poor customerservice, empty transport vehicle on return route, excessive fuel prices,excessive tariffs, and the like.

Referring to FIG. 21 , a management platform of an informationtechnology system, such as a management platform for a value chain ofgoods and/or services is depicted as a block diagram of functionalelements and representative interconnections automated coordination of aset of value chain network activities for a set of products of anenterprise. The management platform includes a set of network-connectedvalue chain network entities 3202 that produce activity information 3208that is used by an artificial intelligence system 1160 to provideautomate coordination 3220 of value chain network activities 3212 for aset of products 3210 for an enterprise 3204. In embodiments, value chainmonitoring systems 614 may monitor activities of the set ofnetwork-connected value chain entities 3202 and work cooperatively withdata collection and management systems 640 to gather and store valuechain entity monitored information, such as activity information,configuration information, and the like. This gathered information maybe configured as activity information 3208 for a set of activitiesassociated with a set of products 3210 of an enterprise 3204. Inembodiments, the artificial intelligence systems 1160 may useapplication programming connectivity facilities 642 for automatingaccess to the monitored activity information 3208.

A value chain may include a plurality of interconnected entities thateach perform several activities for completing the value chain. Whilehumans play a critical role in some activities within a value chainnetwork, greater automated coordination and unified orchestration ofsupply and demand may be achieved using artificial intelligence-typesystems (e.g., machine learning, expert systems, self-organizingsystems, and the like including such systems describe herein and in thedocuments incorporated herein by reference) for coordinating supplychain activities. Use of artificial intelligence may further enrich theemerging nature of self-adapting systems, including Internet of Things(IoT) devices and intelligent products and the like that not onlyprovide greater capabilities to end users, but can play a critical rolein automated coordination of supply chain activities.

For example, an IoT system deployed in a fulfillment center 628 maycoordinate with an intelligent product 1510 that takes customer feedbackabout the product 1510, and an application 630 for the fulfillmentcenter 628 may, upon receiving customer feedback via a connection pathto the intelligent product 1510 about a problem with the product 1510,initiate a workflow to perform corrective actions on similar products650 before the products 650 are sent out from the fulfillment center628. The workflow may be configured by an artificial intelligence system1160 that analyzes the problem with the product 1510, develops anunderstanding of value chain network activities that produce theproduct, determines resources required for the workflow, coordinateswith inventory and production systems to adapt any existing workflowsand the like. Artificial intelligence systems 1160 may furthercoordinate with demand management applications to address any temporaryimpact on product availability and the like.

In embodiments, automated coordination of a set of value chain networkactivities for a set of products for an enterprise may rely on themethods and systems of coordinated intelligence described herein, suchas to facilitate coordinating demand management activities, supply chainactivities and the like, optionally using artificial intelligence forproviding the coordinated intelligence, coordinating the activities andthe like. As an example, artificial intelligence may facilitatedetermining relationships among value change network activities based oninputs used by the activities and results produced by the activities.Artificial intelligence may be integrated with and/or work cooperativelywith activities of the platform, such as value chain network entityactivities to continuously monitor activities, identify temporal aspectsneeding coordination (e.g., when changes in supply temporally impactdemand activities), and automate such coordination. Automatedcoordination of value chain network activities within and across valuechain network entity activities may benefit from advanced artificialintelligence systems that may enable use of differing artificialintelligence capabilities for any given value chain set of entities,applications, or conditions. Use of hybrid artificial intelligencesystems may provide benefits by applying more than one type ofintelligence to a set of conditions to facilitate human and/or computerautomated selection thereof. Artificial intelligence can further enhanceautomated coordination of value chain network entity activities throughintelligent operations such as generating sets of predictions, sets ofclassifications, generation of automate control signals (that may becommunicated across value chain network entities and the like). Otherexemplary artificial intelligence-based influences on automatedcoordination of value chain network entity activities include machinelearning-based information routing and recommendations thereto,semi-sentient problem recognition based on both structured (e.g.,production data) and unstructured (e.g., human emotions) sources, andthe like. Artificial intelligence systems may facilitate automatedcoordination of value chain network entity activities for a set ofproducts or an enterprise based on adaptive intelligence provided by theplatform for a category of goods under which the set of products of anenterprise may be grouped. In an example, adaptive intelligence may beprovided by the platform for a drapery hanging category of goods and aset of products for an enterprise may include a line of adaptabledrapery hangers. Through understanding developed for the overall draperyhanging category, artificial intelligence capabilities may be applied tovalue chain network activities of the enterprise for automating aspectsof the value chain, such as information exchange among activities andthe like.

Digital Twin System in Value Chain Entity Management Platform

Referring to FIG. 22 , the adaptive intelligence layer 614 may include avalue chain network digital twin system 1700, which may include a set ofcomponents, processes, services, interfaces and other elements fordevelopment and deployment of digital twin capabilities forvisualization of various value chain entities 652, environments, andapplications 630, as well as for coordinated intelligence (includingartificial intelligence 1160, edge intelligence 1400, analytics andother capabilities) and other value-added services and capabilities thatare enabled or facilitated with a digital twin 1700. Without limitation,a digital twin 1700 may be used for and/or applied to each of theprocesses that are managed, controlled, or mediated by each of the setof applications 614 of the platform application layer.

In embodiments, the digital twin 1700 may take advantage of the presenceof multiple applications 630 within the value chain management platform604, such that a pair of applications may share data sources (such as inthe data storage layer 624) and other inputs (such as from themonitoring layer 614) that are collected with respect to value chainentities 652, as well as sharing outputs, events, state information andoutputs, which collectively may provide a much richer environment forenriching content in a digital twin 1700, including through use ofartificial intelligence 1160 (including any of the various expertsystems, artificial intelligence systems, neural networks, supervisedlearning systems, machine learning systems, deep learning systems, andother systems described throughout this disclosure and in the documentsincorporated by reference) and through use of content collected by themonitoring layer 614 and data collection systems 640.

In embodiments, a digital twin 1700 may be used in connection withshared or converged processes among the various pairs of theapplications 630 of the application 604, such as, without limitation, ofa converged process involving a security application 834 and aninventory management application 820, integrated automation ofblockchain-based applications 844 with facility management applications850, and many others. In embodiments, converged processes may includeshared data structures for multiple applications 630 (including onesthat track the same transactions on a blockchain but may consumedifferent subsets of available attributes of the data objects maintainedin the blockchain or ones that use a set of nodes and links in a commonknowledge graph) that may be connected to with the digital twin 1700such that the digital twin 1700 is updated accordingly. For example, atransaction indicating a change of ownership of an entity 652 may bestored in a blockchain and used by multiple applications 630, such as toenable role-based access control, role-based permissions for remotecontrol, identity-based event reporting, and the like that may beconnected to and shared with the digital twin 1700 such that the digitaltwin 1700 may be updated accordingly. In embodiments, convergedprocesses may include shared process flows across applications 630,including subsets of larger flows that are involved in one or more of aset of applications 614 that may be connected to and shared with thedigital twin 1700 such that the digital twin 1700 may be updatedaccordingly. For example, an inspection flow about a value chain networkentity 652 may serve an analytics solution 838, an asset managementsolution 814, and others.

In embodiments, a digital twin 1700 may be provided for the wide rangeof value chain network applications 630 mentioned throughout thisdisclosure and the documents incorporated herein by reference. Anenvironment for development of a digital twin 1700 may include a set ofinterfaces for developers in which a developer may configure anartificial intelligence system 1160 to take inputs from selected datasources of the data storage layer 624 and events or other data from themonitoring systems layer 614 and supply them for inclusion in a digitaltwin 1700. A digital twin 1700 development environment may be configuredto take outputs and outcomes from various applications 630.

Value Chain Network Digital Twins

Referring to FIG. 23 , any of the value chain network entities 652 canbe depicted in a set of one or more digital twins 1700, such as bypopulating the digital twin 1700 with value chain network data object1004, such as event data 1034, state data 1140, or other data withrespect to value chain network entities 652, applications 630, orcomponents or elements of the platform 604 as described throughout thisdisclosure.

Thus, the platform 604 may include, integrate, integrate with, manage,control, coordinate with, or otherwise handle any of a wide variety ofdigital twins 1700, such as distribution twins 1714 (such asrepresenting distribution facilities, assets, objects, workers, or thelike); warehousing twins 1712 (such as representing warehousefacilities, assets, objects, workers and the like); port infrastructuretwins 1714 (such as representing a seaport, an airport, or otherfacility, as well as assets, objects, workers and the like); shippingfacility twins 1720; operating facility twins 1722; customer twins 1730(such as representing physical, behavioral, demographic, psychographic,financial, historical, affinity, interest, and other characteristics ofgroups of customers or individual customers); worker twins 1740 (such asrepresenting physical attributes, physiologic data, status data,psychographic information, emotional states, states of fatigue/energy,states of attention, skills, training, competencies, roles, authority,responsibilities, work status, activities, and other attributes of orinvolving workers); wearable/portable device twins 1750; process twins1760; machine twins 21010 (such as for various machines used to supporta value chain network 668); product twins 1780; point of origin twins1560; supplier twins 1630; supply factor twins 1650; maritime facilitytwins 1572; floating asset twins 1570; shipyard twins 1620; destinationtwins 1562; fulfillment twins 1600; delivery system twins 1610; demandfactor twins 1640; retailer twins 1790; ecommerce and online site andoperator twins 1800; waterway twins 1810; roadway twins 1820; railwaytwins 1830; air facility twins 1840 (such as twins of aircraft, runways,airports, hangars, warehouses, air travel routes, refueling facilitiesand other assets, objects, workers and the like used in connection withair transport of products 650); autonomous vehicle twins 1850; roboticstwins 1860; drone twins 1870; and logistics factor twins 1880; amongothers. Each of these may have characteristics of digital twinsdescribed throughout this disclosure and the documents incorporated byreference herein, such as mirroring or reflecting changes in states ofassociated physical objects or other entities, providing capabilitiesfor modeling behavior or interactions of associated physical objects orother entities, enabling simulations, providing indications of status,and many others.

In example embodiments, a digital twin system may be configured togenerate a variety of enterprise digital twins 1700 in connection with avalue chain (e.g., specifically value chain network entities 652). Forexample, an enterprise that produces goods internationally (or atmultiple facilities) may configure a set of digital twins 1700, such assupplier twins that depict the enterprise's supply chain, factory twinsof the various production facilities, product twins that represent theproducts made by the enterprise, distribution twins that represent theenterprise's distribution chains, and other suitable twins. In doing so,the enterprise may define the structural elements of each respectivedigital twin as well as any system data that corresponds to thestructural elements of the digital twin. For instance, in generating aproduction facility twin, the enterprise may the layout and spatialdefinitions of the facility and any processes that are performed in thefacility. The enterprise may also define data sources corresponding tothe value chain network entities 652, such as sensor systems, smartmanufacturing equipment, inventory systems, logistics systems, and thelike that provide data relevant to the facility. The enterprise mayassociate the data sources with elements of the production facilityand/or the processes occurring the facility. Similarly, the enterprisemay define the structural, process, and layout definitions of its supplychain and its distribution chain and may connect relevant data sources,such as supplier databases, logistics platforms, to generate respectivedistribution chain and supply chain twins. The enterprise may furtherassociate these digital twins to have a view of its value chain. Inembodiments, the digital twin system may perform simulations of theenterprise's value chain that incorporate real-time data obtained fromthe various value chain network entities 652 of the enterprise. In someof these embodiments, the digital twin system may recommend decisions toa user interacting with the enterprise digital twins 1700, such as whento order certain parts for manufacturing a certain product given apredicted demand for the manufactured product, when to schedulemaintenance on machinery and/or replace machinery (e.g., when digitalsimulations on the digital twin indicates the demand for certainproducts may be the lowest or when it would have the least effect on theenterprise's profits and losses statement), what time of day to shipitems, or the like. The foregoing example is a non-limiting example ofthe manner by which a digital twin may ingest system data and performsimulations in order to further one or more goals.

Entity Discovery and Interaction Management

Referring to FIG. 24 , the monitoring systems layer 614, includingvarious data collection systems 640 (such as IoT data collectionsystems, data collection systems that search social networks, websites,and other online resources, crowdsourcing systems, and others) mayinclude a set of entity discovery systems 1900, such as for identifyingsets of value chain network entities 652, identifying types of valuechain network entities 652, identifying specific value chain networkentities 652 and the like, as well as for managing identities of thevalue chain network entities 652, including for resolving ambiguities(such as where a single entity is identified differently in differentsystems, where different entities are identified similarly, and thelike), for entity identity deduplication, for entity identityresolution, for entity identity enhancement (such as by enriching dataobjects with additional data that is collected about an entity withinthe platform), and the like. Entity discovery 1900 may also includediscovery of interactions among entities, such as how entities areconnected (e.g., by what network connections, data integration systems,and/or interfaces), what data is exchanged among entities (includingwhat types of data objects are exchanged, what common workflows involveentities, what inputs and outputs are exchanged between entities, andthe like), what rules or policies govern the entities, and the like. Theplatform 604 may include a set of entity interaction management systems1902, which may comprise one or more artificial intelligence systems(including any of the types described throughout this disclosure) formanaging a set of interactions among entities that are discoveredthrough entity discovery 1900, including ones that learn on a trainingset of data to manage interactions among entities based on how entitieshave been managed by human supervisors or by other systems.

As an illustrative example among many possible ones, the entitydiscovery system 1900 may be used to discover a network-connected camerathat shows the loading dock of facility that produces a product for anenterprise, as well as to identify what interfaces or protocols areneeded to access a feed of video content from the camera. The entityinteraction management system 1902 may then be used to interact with theinterfaces or protocols to set up access to the feed and to provide thefeed to another system for further processing, such as to have anartificial intelligence system 1160 process the feed to discoverycontent that is relevant to an activity of the enterprise. For example,the artificial intelligence system 1160 may process image frames of thevideo feed to find markings (such as produce labels, SKUs, images,logos, or the like), shapes (such as packages of a particular size orshape), activities (such as loading or unloading activities) or the likethat may indicate that a product has moved through the loading dock.This information may substitute for, augment, or be used to validateother information, such as RFID tracking information or the like Similardiscovery and interaction management activities may be undertaken withany of the types of value chain network entities 652 describedthroughout this disclosure.

Robotic Process Automation in Value Chain Network

Referring to FIG. 25 , the adaptive intelligence layer 614 may include arobotic process automation (RPA) system 1442, which may include a set ofcomponents, processes, services, interfaces and other elements fordevelopment and deployment of automation capabilities for various valuechain entities 652, environments, and applications 630. Withoutlimitation, robotic process automation 1442 may be applied to each ofthe processes that are managed, controlled, or mediated by each of theset of applications 614 of the platform application layer, to functions,components, workflows, processes of the VCNP 604 itself, to processesinvolving value chain network entities 652 and other processes.

In embodiments, robotic process automation 1442 may take advantage ofthe presence of multiple applications 630 within the value chainmanagement platform 604, such that a pair of applications may share datasources (such as in the data storage layer 624) and other inputs (suchas from the monitoring layer 614) that are collected with respect tovalue chain entities 652, as well as sharing outputs, events, stateinformation and outputs, which collectively may provide a much richerenvironment for process automation, including through use of artificialintelligence 1160 (including any of the various expert systems,artificial intelligence systems, neural networks, supervised learningsystems, machine learning systems, deep learning systems, and othersystems described throughout this disclosure and in the documentsincorporated by reference). For example, an asset management application814 may use robotic process automation 1442 for automation of an assetinspection process that is normally performed or supervised by a human(such as by automating a process involving visual inspection using videoor still images from a camera or other that displays images of an entity652, such as where the robotic process automation 1442 system is trainedto automate the inspection by observing interactions of a set of humaninspectors or supervisors with an interface that is used to identify,diagnose, measure, parameterize, or otherwise characterize possibledefects or favorable characteristics of a facility or other asset. Inembodiments, interactions of the human inspectors or supervisors mayinclude a labeled data set where labels or tags indicate types ofdefects, favorable properties, or other characteristics, such that amachine learning system can learn, using the training data set, toidentify the same characteristics, which in turn can be used to automatethe inspection process such that defects or favorable properties areautomatically classified and detected in a set of video or still images,which in turn can be used within the value chain network assetmanagement application 814 to flag items that require furtherinspection, that should be rejected, that should be disclosed to aprospective buyer, that should be remediated, or the like. Inembodiments, robotic process automation 1442 may involvemulti-application or cross-application sharing of inputs, datastructures, data sources, events, states, outputs or outcomes. Forexample, the asset management application 814 may receive informationfrom a marketplace application 854 that may enrich the robotic processautomation 1442 of the asset management application 814, such asinformation about the current characteristics of an item from aparticular vendor in the supply chain for an asset, which may assist inpopulating the characteristics about the asset for purposes offacilitating an inspection process, a negotiation process, a deliveryprocess, or the like. These and many other examples of multi-applicationor cross-application sharing for robotic process automation 1442 acrossthe applications 630 are encompassed by the present disclosure. Roboticprocess automation 1442 may be used with various functionality of theVCNP 604. For example, in some embodiments, robotic process automation1442 may be described as training a robot to operate and automate a taskthat was, to at least a large extent, governed by a human. One of thesetasks may be used to train a robot that may train other robots. Therobotic process automation 1442 may be trained (e.g., through machinelearning) to mimic interactions on a training set, and then have thistrained robotic process automation 1442 (e.g., trained agent or trainedrobotic process automation system) execute these tasks that werepreviously performed by people. For example, the robotic processautomation 1442 may utilize software that may provide softwareinteraction observations (such as mouse movements, mouse clicks, cursormovements, navigation actions, menu selections, keyboard typing, andmany others), such as logged and/or tracked by software interactionobservation system 1500, purchase of the product by a customer 714, andthe like. This may include monitoring of a user's mouse clicks, mousemovements, and/or keyboard typing to learn to do the same clicks and/ortyping. In another example, the robotic process automation 1442 mayutilize software to learn physical interactions with robots and othersystems to train a robotic system to sequence or undertake the samephysical interactions. For example, the robot may be trained to rebuilda set of bearings by having the robot watch a video of someone doingthis task. This may include tracking physical interactions and trackinginteractions at a software level. The robotic process automation 1442may understand what the underlying competencies are that are beingdeployed such that the VCNP 604 preconfigure combinations of neuralnetworks that may be used to replicate performance of humancapabilities.

In embodiments, robotic process automation may be applied to shared orconverged processes among the various pairs of the applications 630 ofthe application 604, such as, without limitation, of a converged processinvolving a security application 834 and an inventory application 820,integrated automation of blockchain-based applications 844 with vendormanagement applications 832, and many others. In embodiments, convergedprocesses may include shared data structures for multiple applications630 (including ones that track the same transactions on a blockchain butmay consume different subsets of available attributes of the dataobjects maintained in the blockchain or ones that use a set of nodes andlinks in a common knowledge graph). For example, a transactionindicating a change of ownership of an entity 652 may be stored in ablockchain and used by multiple applications 630, such as to enablerole-based access control, role-based permissions for remote control,identity-based event reporting, and the like. In embodiments, convergedprocesses may include shared process flows across applications 630,including subsets of larger flows that are involved in one or more of aset of applications 614. For example, a risk management or inspectionflow about an entity 652 may serve an inventory management application832, an asset management application 814, a demand managementapplication 824, and a supply chain application 812, among others.

In embodiments, robotic process automation 1442 may be provided for thewide range of value chain network processes mentioned throughout thisdisclosure and the documents incorporated herein by reference, includingwithout limitation all of the applications 630. An environment fordevelopment of robotic process automation for value chain networks mayinclude a set of interfaces for developers in which a developer mayconfigure an artificial intelligence system 1160 to take inputs fromselected data sources of the VCN data storage layer 624 and event data1034, state data 1140 or other value chain network data objects 1004from the monitoring systems layer 614 and supply them, such as to aneural network, either as inputs for classification or prediction, or asoutcomes relating to the platform 102, value chain network entities 652,applications 630, or the like. The RPA development environment 1442 maybe configured to take outputs and outcomes 1040 from variousapplications 630, again to facilitate automated learning and improvementof classification, prediction, or the like that is involved in a step ofa process that is intended to be automated. In embodiments, thedevelopment environment, and the resulting robotic process automation1442 may involve monitoring a combination of both software programinteraction observations 1500 (e.g., by workers interacting with varioussoftware interfaces of applications 630 involving value chain networkentities 652) and physical process interaction observations 1510 (e.g.,by watching workers interacting with or using machines, equipment, toolsor the like in a value chain network 668). In embodiments, observationof software interactions 1500 may include interactions among softwarecomponents with other software components, such as how one application630 interacts via APIs with another application 630. In embodiments,observation of physical process interactions 1510 may includeobservation (such as by video cameras, motion detectors, or othersensors, as well as detection of positions, movements, or the like ofhardware, such as robotic hardware) of how human workers interact withvalue chain entities 652 (such as locations of workers (including routestaken through a location, where workers of a given type are locatedduring a given set of events, processes or the like, how workersmanipulate pieces of equipment, cargo, containers, packages, products650 or other items using various tools, equipment, and physicalinterfaces, the timing of worker responses with respect to variousevents (such as responses to alerts and warnings), procedures by whichworkers undertake scheduled deliveries, movements, maintenance, updates,repairs and service processes; procedures by which workers tune oradjust items involved in workflows, and many others). Physical processobservation 1510 may include tracking positions, angles, forces,velocities, acceleration, pressures, torque, and the like of a worker asthe worker operates on hardware, such as on a container or package, oron a piece of equipment involved in handling products, with a tool. Suchobservations may be obtained by any combination of video data, datadetected within a machine (such as of positions of elements of themachine detected and reported by position detectors), data collected bya wearable device (such as an exoskeleton that contains positiondetectors, force detectors, torque detectors and the like that isconfigured to detect the physical characteristics of interactions of ahuman worker with a hardware item for purposes of developing a trainingdata set). By collecting both software interaction observations 1500 andphysical process interaction observations 1510 the RPA system 1442 canmore comprehensively automate processes involving value chain entities652, such as by using software automation in combination with physicalrobots.

In embodiments, robotic process automation 1442 is configured to train aset of physical robots that have hardware elements that facilitateundertaking tasks that are conventionally performed by humans. These mayinclude robots that walk (including walking up and down stairs todeliver a package), climb (such as climbing ladders in a warehouse toreach shelves where products 650 are stored), move about a facility,attach to items, grip items (such as using robotic arms, hands, pincers,or the like), lift items, carry items, remove and replace items, usetools and many others.

Value Chain Management Platform—Unified Robotic Process Automation forDemand Management and Supply Chain

In embodiments, provided herein are methods, systems, components andother elements for an information technology system that may include acloud-based management VCNP 604 with a micro-services architecture, aset of interfaces 702, a set of network connectivity facilities 642,adaptive intelligence facilities 614, data storage facilities 624, datacollection systems 640, and monitoring facilities 614 that arecoordinated for monitoring and management of a set of value chainnetwork entities 652; a set of applications for enabling an enterpriseto manage a set of value chain network entities from a point of originto a point of customer use; and a unified set of robotic processautomation systems 1442 that provide coordinated automation amongvarious applications 630, including demand management applications,supply chain applications, intelligent product applications andenterprise resource management applications for a category of goods.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a unified set of roboticprocess automation systems that provide coordinated automation among atleast two types of applications from among a set of demand managementapplications, a set of supply chain applications, a set of intelligentproduct applications and a set of enterprise resource managementapplications for a category of goods.

Value Chain Management Platform—Robotic Process Automation Services inMicroservices Architecture for Value Chain Network

In embodiments, provided herein are methods, systems, components andother elements for an information technology system that may include acloud-based management VCNP 102 with a micro-services architecture, aset of interfaces 702, a set of network connectivity facilities 642,adaptive intelligence facilities 614, data storage facilities 624, datacollection systems 640, and monitoring facilities 614 that arecoordinated for monitoring and management of a set of value chainnetwork entities 652; a set of applications for enabling an enterpriseto manage a set of value chain network entities from a point of originto a point of customer use; and a set of microservices layers includingan application layer supporting at least one supply chain applicationand at least one demand management application, wherein the microservicelayers include a robotic process automation layer 1442 that usesinformation collected by a data collection layer 640 and a set ofoutcomes and activities 1040 involving the applications of theapplication layer 630 to automate a set of actions for at least a subsetof the applications 630.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of microservices layersincluding an application layer supporting at least one supply chainapplication and at least one demand management application, wherein themicroservice layers include a robotic process automation layer that usesinformation collected by a data collection layer and a set of outcomesand activities involving the applications of the application layer toautomate a set of actions for at least a subset of the applications.

Value Chain Management Platform—Robotic Process Automation for ValueChain Network Processes

In embodiments, provided herein are methods, systems, components andother elements for an information technology system that may include acloud-based management VCNP 102 with a micro-services architecture, aset of interfaces 702, a set of network connectivity facilities 642,adaptive intelligence facilities 614, data storage facilities 624, datacollection systems 640, and monitoring facilities 614 that arecoordinated for monitoring and management of a set of value chainnetwork entities 652; a set of applications for enabling an enterpriseto manage a set of value chain network entities from a point of originto a point of customer use; and a set of robotic process automationsystems 1442 for automating a set of processes in a value chain network,wherein the robotic process automation systems 1442 learn on a trainingset of data involving a set of user interactions with a set ofinterfaces 702 of a set of software systems that are used to monitor andmanage the value chain network entities 652, as well as from variousprocess and application outputs and outcomes 1040 that may occur with orwithin the VCNP 102.

In embodiments, the value chain network entities 652 may include, forexample, products, suppliers, producers, manufacturers, retailers,businesses, owners, operators, operating facilities, customers,consumers, workers, mobile devices, wearable devices, distributors,resellers, supply chain infrastructure facilities, supply chainprocesses, logistics processes, reverse logistics processes, demandprediction processes, demand management processes, demand aggregationprocesses, machines, ships, barges, warehouses, maritime ports,airports, airways, waterways, roadways, railways, bridges, tunnels,online retailers, ecommerce sites, demand factors, supply factors,delivery systems, floating assets, points of origin, points ofdestination, points of storage, points of use, networks, informationtechnology systems, software platforms, distribution centers,fulfillment centers, containers, container handling facilities, customs,export control, border control, drones, robots, autonomous vehicles,hauling facilities, drones/robots/AVs, waterways, port infrastructurefacilities, or many others.

In embodiments, the robotic process automation layer automates a processthat may include, for example, without limitation, selection of aquantity of product for an order, selection of a carrier for a shipment,selection of a vendor for a component, selection of a vendor for afinished goods order, selection of a variation of a product formarketing, selection of an assortment of goods for a shelf,determination of a price for a finished good, configuration of a serviceoffer related to a product, configuration of product bundle,configuration of a product kit, configuration of a product package,configuration of a product display, configuration of a product image,configuration of a product description, configuration of a websitenavigation path related to a product, determination of an inventorylevel for a product, selection of a logistics type, configuration of aschedule for product delivery, configuration of a logistics schedule,configuration of a set of inputs for machine learning, preparation ofproduct documentation, preparation of required disclosures about aproduct, configuration of a product for a set of local requirements,configuration of a set of products for compatibility, configuration of arequest for proposals, ordering of equipment for a warehouse, orderingof equipment for a fulfillment center, classification of a productdefect in an image, inspection of a product in an image, inspection ofproduct quality data from a set of sensors, inspection of data from aset of onboard diagnostics on a product, inspection of diagnostic datafrom an Internet of Things system, review of sensor data fromenvironmental sensors in a set of supply chain environments, selectionof inputs for a digital twin, selection of outputs from a digital twin,selection of visual elements for presentation in a digital twin,diagnosis of sources of delay in a supply chain, diagnosis of sources ofscarcity in a supply chain, diagnosis of sources of congestion in asupply chain, diagnosis of sources of cost overruns in a supply chain,diagnosis of sources of product defects in a supply chain, prediction ofmaintenance requirements in supply chain infrastructure, or others.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; and a set of robotic process automationsystems for automating a set of processes in a value chain network,wherein the robotic process automation systems learn on a training setof data involving a set of user interactions with a set of interfaces ofa set of software systems that are used to monitor and manage the valuechain network entities.

In embodiments, one of the processes automated by robotic processautomation as described in any of the embodiments disclosed herein mayinvolve the following. In embodiments, RPA involves selection of aquantity of product for an order. In embodiments, one of the processesautomated by robotic process automation involves selection of a carrierfor a shipment. In embodiments, one of the processes automated byrobotic process automation involves selection of a vendor for acomponent. In embodiments, one of the processes automated by roboticprocess automation involves selection of a vendor for a finished goodsorder. In embodiments, one of the processes automated by robotic processautomation involves selection of a variation of a product for marketing.In embodiments, one of the processes automated by robotic processautomation involves selection of an assortment of goods for a shelf. Inembodiments, one of the processes automated by robotic processautomation involves determination of a price for a finished good. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a service offer related to aproduct. In embodiments, one of the processes automated by roboticprocess automation involves configuration of product bundle. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a product kit. In embodiments, oneof the processes automated by robotic process automation involvesconfiguration of a product package. In embodiments, one of the processesautomated by robotic process automation involves configuration of aproduct display. In embodiments, one of the processes automated byrobotic process automation involves configuration of a product image. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a product description. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a website navigation path relatedto a product. In embodiments, one of the processes automated by roboticprocess automation involves determination of an inventory level for aproduct. In embodiments, one of the processes automated by roboticprocess automation involves selection of a logistics type. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a schedule for product delivery. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a logistics schedule. Inembodiments, one of the processes automated by robotic processautomation involves configuration of a set of inputs for machinelearning. In embodiments, one of the processes automated by roboticprocess automation involves preparation of product documentation. Inembodiments, one of the processes automated by robotic processautomation involves preparation of required disclosures about a product.In embodiments, one of the processes automated by robotic processautomation involves configuration of a product for a set of localrequirements. In embodiments, one of the processes automated by roboticprocess automation involves configuration of a set of products forcompatibility. In embodiments, one of the processes automated by roboticprocess automation involves configuration of a request for proposals.

In embodiments, one of the processes automated by robotic processautomation involves ordering of equipment for a warehouse. Inembodiments, one of the processes automated by robotic processautomation involves ordering of equipment for a fulfillment center. Inembodiments, one of the processes automated by robotic processautomation involves classification of a product defect in an image. Inembodiments, one of the processes automated by robotic processautomation involves inspection of a product in an image.

In embodiments, one of the processes automated by robotic processautomation involves inspection of product quality data from a set ofsensors. In embodiments, one of the processes automated by roboticprocess automation involves inspection of data from a set of onboarddiagnostics on a product. In embodiments, one of the processes automatedby robotic process automation involves inspection of diagnostic datafrom an Internet of Things system. In embodiments, one of the processesautomated by robotic process automation involves review of sensor datafrom environmental sensors in a set of supply chain environments.

In embodiments, one of the processes automated by robotic processautomation involves selection of inputs for a digital twin. Inembodiments, one of the processes automated by robotic processautomation involves selection of outputs from a digital twin. Inembodiments, one of the processes automated by robotic processautomation involves selection of visual elements for presentation in adigital twin. In embodiments, one of the processes automated by roboticprocess automation involves diagnosis of sources of delay in a supplychain. In embodiments, one of the processes automated by robotic processautomation involves diagnosis of sources of scarcity in a supply chain.In embodiments, one of the processes automated by robotic processautomation involves diagnosis of sources of congestion in a supplychain.

In embodiments, one of the processes automated by robotic processautomation involves diagnosis of sources of cost overruns in a supplychain. In embodiments, one of the processes automated by robotic processautomation involves diagnosis of sources of product defects in a supplychain. In embodiments, one of the processes automated by robotic processautomation involves prediction of maintenance requirements in supplychain infrastructure.

In embodiments, the set of demand management applications, supply chainapplications, intelligent product applications and enterprise resourcemanagement applications may include, for example, ones involving supplychain, asset management, risk management, inventory management, demandmanagement, demand prediction, demand aggregation, pricing, positioning,placement, promotion, blockchain, smart contract, infrastructuremanagement, facility management, analytics, finance, trading, tax,regulatory, identity management, commerce, ecommerce, payments,security, safety, vendor management, process management, compatibilitytesting, compatibility management, infrastructure testing, incidentmanagement, predictive maintenance, logistics, monitoring, remotecontrol, automation, self-configuration, self-healing,self-organization, logistics, reverse logistics, waste reduction,augmented reality, virtual reality, mixed reality, demand customerprofiling, entity profiling, enterprise profiling, worker profiling,workforce profiling, component supply policy management, product design,product configuration, product updating, product maintenance, productsupport, product testing, warehousing, distribution, fulfillment, kitconfiguration, kit deployment, kit support, kit updating, kitmaintenance, kit modification, kit management, shipping fleetmanagement, vehicle fleet management, workforce management, maritimefleet management, navigation, routing, shipping management, opportunitymatching, search, advertisement, entity discovery, entity search,distribution, delivery, enterprise resource planning, and many others.

Opportunity Miners for Automated Improvement of Adaptive Intelligence

Referring to FIG. 26 , a set of opportunity miners 1460 may be providedas part of the adaptive intelligence layer 614, which may be configuredto seek and recommend opportunities to improve one or more of theelements of the platform 604, such as via addition of artificialintelligence 1160, automation (including robotic process automation1442), or the like to one or more of the systems, sub-systems,components, applications or the like of the VCNP 102 or with which theVCNP 102 interacts. In embodiments, the opportunity miners 1460 may beconfigured or used by developers of AI or RPA solutions to findopportunities for better solutions and to optimize existing solutions ina value chain network 668. In embodiments, the opportunity miners 1460may include a set of systems that collect information within the VCNP102 and collect information within, about and for a set of value chainnetwork entities 652 and environments, where the collected informationhas the potential to help identify and prioritize opportunities forincreased automation and/or intelligence about the value chain network668, about applications 630, about value chain network entities 652, orabout the VCNP 102 itself. For example, the opportunity miners 1460 mayinclude systems that observe clusters of value chain network workers bytime, by type, and by location, such as using cameras, wearables, orother sensors, such as to identify labor-intensive areas and processesin a set of value chain network 668 environments. These may bepresented, such as in a ranked or prioritized list, or in avisualization (such as a heat map showing dwell times of customers,workers or other individuals on a map of an environment or a heat mapshowing routes traveled by customers or workers within an environment)to show places with high labor activity. In embodiments, analytics 838may be used to identify which environments or activities would mostbenefit from automation for purposes of improved delivery times,mitigation of congestion, and other performance improvements.

In embodiments, opportunity mining may include facilities forsolicitation of appropriate training data sets that may be used tofacilitate process automation. For example, certain kinds of inputs, ifavailable, would provide very high value for automation, such as videodata sets that capture very experienced and/or highly expert workersperforming complex tasks. Opportunity miners 1460 may search for suchvideo data sets as described herein; however, in the absence of success(or to supplement available data), the platform may include systems bywhich a user, such as a developer, may specify a desired type of data,such as software interaction data (such as of an expert working with aprogram to perform a particular task), video data (such as video showinga set of experts performing a certain kind of delivery process, packingprocess, picking process, a container movement process, or the like),and/or physical process observation data (such as video, sensor data, orthe like). The resulting library of interactions captured in response tospecification may be captured as a data set in the data storage layer624, such as for consumption by various applications 630, adaptiveintelligence systems 614, and other processes and systems. Inembodiments, the library may include videos that are specificallydeveloped as instructional videos, such as to facilitate developing anautomation map that can follow instructions in the video, such asproviding a sequence of steps according to a procedure or protocol,breaking down the procedure or protocol into sub-steps that arecandidates for automation, and the like. In embodiments, such videos maybe processed by natural language processing, such as to automaticallydevelop a sequence of labeled instructions that can be used by adeveloper to facilitate a map, a graph, or other models of a processthat assists with development of automation for the process. Inembodiments, a specified set of training data sets may be configured tooperate as inputs to learning. In such cases the training data may betime-synchronized with other data within the platform 604, such asoutputs and outcomes from applications 630, outputs and outcomes ofvalue chain entities 652, or the like, so that a given video of aprocess can be associated with those outputs and outcomes, therebyenabling feedback on learning that is sensitive to the outcomes thatoccurred when a given process that was captured (such as on video, orthrough observation of software interactions or physical processinteractions). For example, this may relate to an instruction video suchas a video of a person who may be building or rebuilding (e.g.,rebuilding a bearing set). This instruction video may include individualsteps for rebuild that may allow a staging of the training to provideinstructions such as parsing the video into stages that mimic theexperts staging in the video. For example, this may include tagging ofthe video to include references to each stage and status (e.g., stageone complete, stage two, etc.) This type of example may utilizeartificial intelligence that may understand that there may be a seriesof sub-functions that add up to a final function.

In embodiments, opportunity miners 1460 may include methods, systems,processes, components, services and other elements for mining foropportunities for smart contract definition, formation, configurationand execution. Data collected within the platform 604, such as any datahandled by the data handling layers 608, stored by the data storagelayer 624, collected by the monitoring layer 614 and collection systems640, collected about or from entities 652 or obtained from externalsources may be used to recognize beneficial opportunities forapplication or configuration of smart contracts. For example, pricinginformation about an entity 652, handled by a pricing application 842,or otherwise collected, may be used to recognize situations in which thesame item or items is disparately priced (in a spot market, futuresmarket, or the like), and the opportunity miner 1460 may provide analert indicating an opportunity for smart contract formation, such as acontract to buy in one environment at a price below a given thresholdand sell in another environment at a price above a given threshold, orvice versa.

In some examples, as shown in FIG. 26 , the adaptive intelligent systems614 may include value translators 1470. The value translators 1470 mayrelate to demand side of transactions. Specifically, for example, thevalue translators 1470 may understand negative currencies of twomarketplaces and may be able to translate value currencies into othercurrencies (e.g., not only fiat currencies that already have cleartranslation functions). In some examples, value translators 1470 may beassociated with points of a point-based system (e.g., in a cost-basedrouting system). In an example embodiment, value translators 1470 may beloyalty points offered that may be convertible into airline seats and/ormay translate to refund policies for staying in a hotel room. In someexamples, different types of entities may be connected as having nativepricing or cost functions that do not always use the same currency orany currency. In another example, value translators 1470 may be usedwith network prioritization or cost-based routing that happens innetworks off of priorities where the point system in these cost-basedrouting systems is not monetary-based.

Broad Management Platform

Referring to FIG. 28 , additional details of an embodiment of theplatform 604 are provided, in particular relating to an overallarchitecture for the platform 604. These may include, for thecloud-based management platform 604, employing a micro-servicesarchitecture, a set of network connectivity facilities 642 (which mayinclude or connect to a set of interfaces 702 of various layers of theplatform 604), a set of adaptive intelligence facilities or adaptiveintelligent systems 614, a set of data storage facilities or systems624, and a set of monitoring facilities or systems 808. The platform 604may support a set of applications 614 (including processes, workflows,activities, events, use cases and applications) for enabling anenterprise to manage a set of value chain network entities 652, such asfrom a point of origin to a point of customer use of a product 1510,which may be an intelligent product.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture; aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities; and a set of applications for enabling an enterprise tomanage a set of value chain network entities from a point of origin to apoint of customer use.

Also provided herein are methods, systems, components and other elementsfor an information technology system that may include: a cloud-basedmanagement platform with a micro-services architecture, the platformhaving: a set of interfaces for accessing and configuring features ofthe platform; a set of network connectivity facilities for enabling aset of value chain network entities to connect to the platform; a set ofadaptive intelligence facilities for automating a set of capabilities ofthe platform; a set of data storage facilities for storing datacollected and handled by the platform; and a set of monitoringfacilities for monitoring the value chain network entities; wherein theplatform hosts a set of applications for enabling an enterprise tomanage a set of value chain network entities from a point of origin of aproduct of the enterprise to a point of customer use.

Broad Management Platform—Details

Referring to FIG. 29 , additional details of an embodiment of theplatform 604 are provided, in particular relating to an overallarchitecture for the platform 604. These may include, for thecloud-based management platform 604, employing a micro-servicesarchitecture, a set of network connectivity facilities 642 (which mayinclude or connect to a set of interfaces 702 of various layers of theplatform 604), a set of adaptive intelligence facilities or adaptiveintelligent systems 614, a set of data storage facilities or systems624, and a set of monitoring facilities or systems 808. The platform 604may support a set of applications 614 (including processes, workflows,activities, events, use cases and applications) for enabling anenterprise to manage a set of value chain network entities 652, such asfrom a point of origin to a point of customer use of a product 1510,which may be an intelligent product.

In embodiments, the set of interfaces 702 may include a demandmanagement interface 1402 and a supply chain management interface 1404.

In embodiments, the set of network connectivity facilities 642 forenabling a set of value chain network entities 652 to connect to theplatform 604 may include a 5G network system 1410, such as one that isdeployed in a supply chain infrastructure facility operated by theenterprise.

In embodiments, the set of network connectivity facilities 642 forenabling a set of value chain network entities 652 to connect to theplatform 604 may include an Internet of Things system 1172, such as onethat is deployed in a supply chain infrastructure facility operated bythe enterprise, in, on or near a value chain network entity 652, in anetwork system, and/or in a cloud computing environment (such as wheredata collection systems 640 are configured to collect and organize IoTdata).

In embodiments, the set of network connectivity facilities 642 forenabling a set of value chain network entities 652 to connect to theVCNP 102 may include a cognitive networking system 1420 deployed in asupply chain infrastructure facility operated by the enterprise.

In embodiments, the set of network connectivity facilities 642 forenabling a set of value chain network entities 652 to connect to theVCNP 102 may include a peer-to-peer network system 1430, such as onethat is deployed in a supply chain infrastructure facility operated bythe enterprise.

In embodiments, the set of adaptive intelligence facilities or adaptiveintelligent systems 614 for automating a set of capabilities of theplatform 604 may include an edge intelligence system 1420, such as onethat is deployed in a supply chain infrastructure facility operated bythe enterprise.

In embodiments, the set of adaptive intelligence facilities or adaptiveintelligent systems 614 for automating a set of capabilities of theplatform 604 may include a robotic process automation system 1442.

In embodiments, the set of adaptive intelligence facilities or adaptiveintelligent systems 614 for automating a set of capabilities of theplatform 604 may include or may integrate with a self-configuring datacollection system 1440, such as one that deployed in a supply chaininfrastructure facility operated by the enterprise, one that is deployedin a network, and/or one that is deployed in a cloud computingenvironment. This may include elements of the data collection systems640 of the data handling layers 608 that interact with or integrate withelements of the adaptive intelligent systems 614.

In embodiments, the set of adaptive intelligence facilities or adaptiveintelligent systems 614 for automating a set of capabilities of theplatform 604 may include a digital twin system 1700, such as onerepresenting attributes of a set of value chain network entities, suchas the ones controlled by an enterprise.

In embodiments, the set of adaptive intelligence facilities or adaptiveintelligent systems 614 for automating a set of capabilities of theplatform 604 may include a smart contract system 848, such as one forautomating a set of interactions or transactions among a set of valuechain network entities 652 based on status data, event data, or otherdata handled by the data handling layers 608.

In embodiments, the set of data storage facilities or data storagesystems 624 for storing data collected and handled by the platform 604uses a distributed data architecture 1122.

In embodiments, the set of data storage facilities for storing datacollected and handled by the platform uses a blockchain 844.

In embodiments, the set of data storage facilities for storing datacollected and handled by the platform uses a distributed ledger 1452.

In embodiments, the set of data storage facilities for storing datacollected and handled by the platform uses graph database 1124representing a set of hierarchical relationships of value chain networkentities.

In embodiments, the set of monitoring facilities 614 for monitoring thevalue chain network entities 652 includes an Internet of Thingsmonitoring system 1172, such as for collecting data from IoT systems anddevices deployed throughout a value chain network.

In embodiments, the set of monitoring facilities 614 for monitoring thevalue chain network entities 652 includes a set of sensor systems 1462,such as ones deployed in a value chain environment or in, one or near avalue chain network entity 652, such as in or on a product 1510.

In embodiments, the set of applications 614 includes a set ofapplications, which may include a variety of types from among, forexample, a set of supply chain management applications 21004, demandmanagement applications 1502, intelligent product applications 1510 andenterprise resource management applications 1520.

In embodiments, the set of applications includes an asset managementapplication 1530.

In embodiments, the value chain network entities 652 as mentionedthroughout this disclosure may include, for example, without limitation,products, suppliers, producers, manufacturers, retailers, businesses,owners, operators, operating facilities, customers, consumers, workers,mobile devices, wearable devices, distributors, resellers, supply chaininfrastructure facilities, supply chain processes, logistics processes,reverse logistics processes, demand prediction processes, demandmanagement processes, demand aggregation processes, machines, ships,barges, warehouses, maritime ports, airports, airways, waterways,roadways, railways, bridges, tunnels, online retailers, ecommerce sites,demand factors, supply factors, delivery systems, floating assets,points of origin, points of destination, points of storage, points ofuse, networks, information technology systems, software platforms,distribution centers, fulfillment centers, containers, containerhandling facilities, customs, export control, border control, drones,robots, autonomous vehicles, hauling facilities, drones/robots/AVs,waterways, port infrastructure facilities, or others.

In embodiments, the platform 604 manages a set of demand factors 1540, aset of supply factors 1550 and a set of value chain infrastructurefacilities 1560.

In embodiments, the supply factors 1550 as mentioned throughout thisdisclosure may include, for example and without limitation, onesinvolving component availability, material availability, componentlocation, material location, component pricing, material pricing,taxation, tariff, impost, duty, import regulation, export regulation,border control, trade regulation, customs, navigation, traffic,congestion, vehicle capacity, ship capacity, container capacity, packagecapacity, vehicle availability, ship availability, containeravailability, package availability, vehicle location, ship location,container location, port location, port availability, port capacity,storage availability, storage capacity, warehouse availability,warehouse capacity, fulfillment center location, fulfillment centeravailability, fulfillment center capacity, asset owner identity, systemcompatibility, worker availability, worker competency, worker location,goods pricing, fuel pricing, energy pricing, route availability, routedistance, route cost, route safety, and many others.

In embodiments, the demand factors 1540 as mentioned throughout thisdisclosure may include, for example and without limitation, onesinvolving product availability, product pricing, delivery timing, needfor refill, need for replacement, manufacturer recall, need for upgrade,need for maintenance, need for update, need for repair, need forconsumable, taste, preference, inferred need, inferred want, groupdemand, individual demand, family demand, business demand, need forworkflow, need for process, need for procedure, need for treatment, needfor improvement, need for diagnosis, compatibility to system,compatibility to product, compatibility to style, compatibility tobrand, demographic, psychographic, geolocation, indoor location,destination, route, home location, visit location, workplace location,business location, personality, mood, emotion, customer behavior,business type, business activity, personal activity, wealth, income,purchasing history, shopping history, search history, engagementhistory, clickstream history, website history, online navigationhistory, group behavior, family behavior, family membership, customeridentity, group identity, business identity, customer profile, businessprofile, group profile, family profile, declared interest, inferredinterest, and many others.

In embodiments, the supply chain infrastructure facilities 1560 asmentioned throughout this disclosure may include, for example andwithout limitation, ship, container ship, boat, barge, maritime port,crane, container, container handling, shipyard, maritime dock,warehouse, distribution, fulfillment, fueling, refueling, nuclearrefueling, waste removal, food supply, beverage supply, drone, robot,autonomous vehicle, aircraft, automotive, truck, train, lift, forklift,hauling facilities, conveyor, loading dock, waterway, bridge, tunnel,airport, depot, vehicle station, train station, weigh station,inspection, roadway, railway, highway, customs house, border control,and other facilities.

In embodiments, the set of applications 614 as mentioned throughout thisdisclosure may include, for example and without limitation, supplychain, asset management, risk management, inventory management, demandmanagement, demand prediction, demand aggregation, pricing, positioning,placement, promotion, blockchain, smart contract, infrastructuremanagement, facility management, analytics, finance, trading, tax,regulatory, identity management, commerce, ecommerce, payments,security, safety, vendor management, process management, compatibilitytesting, compatibility management, infrastructure testing, incidentmanagement, predictive maintenance, logistics, monitoring, remotecontrol, automation, self-configuration, self-healing,self-organization, logistics, reverse logistics, waste reduction,augmented reality, virtual reality, mixed reality, demand customerprofiling, entity profiling, enterprise profiling, worker profiling,workforce profiling, component supply policy management, product design,product configuration, product updating, product maintenance, productsupport, product testing, warehousing, distribution, fulfillment, kitconfiguration, kit deployment, kit support, kit updating, kitmaintenance, kit modification, kit management, shipping fleetmanagement, vehicle fleet management, workforce management, maritimefleet management, navigation, routing, shipping management, opportunitymatching, search, advertisement, entity discovery, entity search,distribution, delivery, enterprise resource planning and otherapplications.

Control Tower

Referring to FIG. 30 , an embodiment of the platform 604 is provided.The platform 604 may employ a micro-services architecture with thevarious data handling layers 608, a set of network connectivityfacilities 642 (which may include or connect to a set of interfaces 702of various layers of the platform 604), a set of adaptive intelligencefacilities or adaptive intelligent systems 614, a set of data storagefacilities or systems 624, and a set of monitoring facilities or systems808. The platform 604 may support a set of applications 614 (includingprocesses, workflows, activities, events, use cases and applications)for enabling an enterprise to manage a set of value chain networkentities 652, such as from a point of origin to a point of customer useof a product 1510, which may be an intelligent product.

In embodiments, the platform 604 may include a user interface 1570 thatprovides a set of unified views for a set of demand managementinformation and supply chain information for a category of goods, suchas one that displays status information, event information, activityinformation, analytics, reporting, or other elements of, relating to, orproduced by a set of supply chain management applications 21004, demandmanagement applications 1502, intelligent product applications 1510 andenterprise resource management applications 1520 that monitor and/ormanage a value chain network and a set of value chain network entities652. The unified view interface 1570 may thus provide, in embodiments, acontrol tower for an enterprise over a range of assets, such as supplychain infrastructure facilities 1560 and other value chain networkentities 652 that are involved as a product 1510 travels from a point oforigin through distribution and retail channels to an environment whereit is used by a customer. These may include views of demand factors 1540and supply factors 1550, so that a user may develop insights aboutconnections among the factors and control one or both of them withcoordinated intelligence. Population of a set of unified views may beadapted over time, such as by learning on outcomes 1040 or otheroperations of the adaptive intelligent systems 614, such as to determinewhich views of the interface 1570 provide the most impactful insights,control features, or the like.

In embodiments, the user interface includes a voice operated assistant1580.

In embodiments, the user interface includes a set of digital twins 1700for presenting a visual representation of a set of attributes of a setof value chain network entities 652.

In embodiments, the user interface 1570 may include capabilities forconfiguring the adaptive intelligent systems 614 or adaptiveintelligence facilities, such as to allow user selection of attributes,parameters, data sources, inputs to learning, feedback to learning,views, formats, arrangements, or other elements.

Value Chain Management Platform—Control Tower UI for Demand Managementand Supply Chain

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a user interface that providesa set of unified views for a set of demand management information andsupply chain information for a category of goods.

Unified Database

Referring to FIG. 31 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the platform 604 may include a unified database 1590that supports a set of applications of multiple types, such as onesamong a set of supply chain management applications 21004, demandmanagement applications 1502, intelligent product applications 1510 andenterprise resource management applications 1520 that monitor and/ormanage a value chain network and a set of value chain network entities652. The unified database 1590 may thus provide, in embodiments,unification of data storage, access and handling for an enterprise overa range of assets, such as supply chain infrastructure facilities 1560and other value chain network entities 652 that are involved as aproduct 1510 travels from a point of origin through distribution andretail channels to an environment where it is used by a customer. Thisunification may provide a number of advantages, including reduced needfor data entry, consistency across applications 630, reduced latency(and better real-time reporting), reduced need for data transformationand integration, and others. These may include data relating to demandfactors 1540 and supply factors 1550, so that an application 630 maybenefit from information collected by, processed, or produced by otherapplications 630 of the platform 604 and a user can develop insightsabout connections among the factors and control one or both of them withcoordinated intelligence. Population of the unified database 1590 may beadapted over time, such as by learning on outcomes 1040 or otheroperations of the adaptive intelligent systems 614, such as to determinewhich elements of the database 1590 should be made available to whichapplications, what data structures provide the most benefit, what datashould be stored or cached for immediate retrieval, what data can bediscarded versus saved, what data is most beneficial to support adaptiveintelligent systems 614, and for other uses.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a unified database thatsupports a set of applications of at least two types from among a set ofdemand management applications, a set of supply chain applications, aset of intelligent product applications and a set of enterprise resourcemanagement applications for a category of goods.

In embodiments, the unified database that supports a set of demandmanagement applications, a set of supply chain applications, a set ofintelligent product applications and a set of enterprise resourcemanagement applications for a category of goods is a distributeddatabase.

In embodiments, the unified database that supports a set of demandmanagement applications, a set of supply chain applications, a set ofintelligent product applications and a set of enterprise resourcemanagement applications for a category of goods uses a graph databasearchitecture. In embodiments, the set of demand management applicationsincludes a demand prediction application. In embodiments, the set ofdemand management applications includes a demand aggregationapplication. In embodiments, the set of demand management applicationsincludes a demand activation application.

In embodiments, the set of supply chain management applications includesa vendor search application. In embodiments, the set of supply chainmanagement applications includes a route configuration application. Inembodiments, the set of supply chain management applications includes alogistics scheduling application.

Unified Data Collection Systems

Referring to FIG. 32 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the platform 604 may include a set of unified set ofdata collection and management systems 640 of the set of monitoringfacilities or systems 808 that support a set of applications 614 ofvarious types, including a set of supply chain management applications21004, demand management applications 1502, intelligent productapplications 1510 and enterprise resource management applications 1520that monitor and/or manage a value chain network and a set of valuechain network entities 652. The unified data collection and managementsystems 640 may thus provide, in embodiments, unification of datamonitoring, search, discovery, collection, access and handling for anenterprise or other user over a range of assets, such as supply chaininfrastructure facilities 1560 and other value chain network entities652 that are involved as a product 1510 travels from a point of originthrough distribution and retail channels to an environment where it isused by a customer. This unification may provide a number of advantages,including reduced need for data entry, consistency across applications630, reduced latency (and better real-time reporting), reduced need fordata transformation and integration, and others. These may includecollection of data relating to demand factors 1540 and supply factors1550, so that an application 630 may benefit from information collectedby, processed, or produced by other applications 630 of the platform 604and a user can develop insights about connections among the factors andcontrol one or both of them with coordinated intelligence. The unifieddata collection and management systems 640 may be adapted over time,such as by learning on outcomes 1040 or other operations of the adaptiveintelligent systems 614, such as to determine which elements of the datacollection and management systems 640 should be made available to whichapplications 630, what data types or sources provide the most benefit,what data should be stored or cached for immediate retrieval, what datacan be discarded versus saved, what data is most beneficial to supportadaptive intelligent systems 614, and for other uses. In exampleembodiments, the unified data collection and management systems 640 mayuse a unified data schema which relates data collection and managementfor various applications. This may be a single point of truth databaseat the most tightly bound or a set of distributed data systems that mayfollow a schema that may be sufficiently common enough that a widevariety of applications may consume the same data as received. Forexample, sensor data may be pulled from a smart product that may beconsumed by a logistics application, a financial application, a demandprediction application, or a genetic programming artificial intelligence(AI) application to change the product, and the like. All of theseapplications may consume data from a data framework. In an example, thismay occur from blockchains that may contain a distributed ledger ortransactional data for purchase and sales or blockchains where there maybe an indication of whether or not events had occurred. In some exampleembodiments, as data moves through a supply chain, this data flow mayoccur through distributed databases, relational databases, graphdatabases of all types, and the like that may be part of the unifieddata collection and management systems 640. In other examples, theunified data collection and management systems 640 may utilize memorythat may be dedicated memory on an asset, in a tag or part of a memorystructure of the device itself that may come from a robust pipeline tiedto the value chain network entities. In other examples, the unified datacollection and management systems 640 may use classic data integrationcapabilities that may include adapting protocols such that they canultimately get to the unified system or schema.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a unified set of datacollection systems that support a set of applications of at least twotypes from among a set of demand management applications, a set ofsupply chain applications, a set of intelligent product applications anda set of enterprise resource management applications for a category ofgoods.

In embodiments, the unified set of data collection systems includes aset of crowdsourcing data collection systems. In embodiments, theunified set of data collection systems includes a set of Internet ofThings data collection systems. In embodiments, the unified set of datacollection systems includes a set of self-configuring sensor systems. Inembodiments, the unified set of data collection systems includes a setof data collection systems that interact with a network-connectedproduct.

In embodiments, the unified set of data collection systems includes aset of mobile data collectors deployed in a set of value chain networkenvironments operated by an enterprise. In embodiments, the unified setof data collection systems includes a set of edge intelligence systemsdeployed in set of value chain network environments operated by anenterprise. In embodiments, the unified set of data collection systemsincludes a set of crowdsourcing data collection systems. In embodiments,the unified set of data collection systems includes a set of Internet ofThings data collection systems. In embodiments, the unified set of datacollection systems includes a set of self-configuring sensor systems. Inembodiments, the unified set of data collection systems includes a setof data collection systems that interact with a network-connectedproduct. In embodiments, the unified set of data collection systemsincludes a set of mobile data collectors deployed in a set of valuechain network environments operated by an enterprise. In embodiments,the unified set of data collection systems includes a set of edgeintelligence systems deployed in a set of value chain networkenvironments operated by an enterprise.

Unified IoT Monitoring Systems

Referring to FIG. 33 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the platform 604 may include a unified set of Internetof Things systems 1172 that provide coordinated monitoring of variousvalue chain entities 652 in service of a set of multiple applications630 of various types, such as a set of supply chain managementapplications 21004, demand management applications 1502, intelligentproduct applications 1510 and enterprise resource managementapplications 1520 that monitor and/or manage a value chain network and aset of value chain network entities 652.

The unified set of Internet of Things systems 1172 may thus provide, inembodiments, unification of monitoring of, and communication with, awide range of facilities, devices, systems, environments, and assets,such as supply chain infrastructure facilities 1560 and other valuechain network entities 652 that are involved as a product 1510 travelsfrom a point of origin through distribution and retail channels to anenvironment where it is used by a customer. This unification may providea number of advantages, including reduced need for data entry,consistency across applications 630, reduced latency, real-timereporting and awareness, reduced need for data transformation andintegration, and others. These may include Internet of Things systems1172 that are used in connection with demand factors 1540 and supplyfactors 1550, so that an application 630 may benefit from informationcollected by, processed, or produced by the unified set of Internet ofThings systems 1172 for other applications 630 of the platform 604, anda user can develop insights about connections among the factors andcontrol one or both of them with coordinated intelligence. The unifiedset of Internet of Things systems 1172 may be adapted over time, such asby learning on outcomes 1040 or other operations of the adaptiveintelligent systems 614, such as to determine which elements of theunified set of Internet of Things systems 1172 should be made availableto which applications 630, what IoT systems 1172 provide the mostbenefit, what data should be stored or cached for immediate retrieval,what data can be discarded versus saved, what data is most beneficial tosupport adaptive intelligent systems 614, and for other uses. In someexamples, the unified set of Internet of Things (IoT) systems 1172 maybe IoT devices that may be installed in various environments. One goalof the unified set of Internet of Things systems 1172 may becoordination across a city or town involving citywide deployments wherecollectively a set of IOT devices may be connected by wide area networkprotocols (e.g., longer range protocols). In another example, theunified set of Internet of Things systems 1172 may involve connecting amesh of devices across several different distribution facilities. TheIoT devices may identify collection for each warehouse and thewarehouses may use the IoT devices to communicate with each other. TheIoT devices may be configured to process data without using the cloud.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications integrated withthe platform for enabling an enterprise user of the platform to manage aset of value chain network entities from a point of origin to a point ofcustomer use; and a unified set of Internet of Things systems thatprovide coordinated monitoring of a set of applications of at least twotypes from among a set of demand management applications, a set ofsupply chain applications, a set of intelligent product applications anda set of enterprise resource management applications for a category ofgoods.

In embodiments, the unified set of Internet of Things systems includes aset of smart home Internet of Things devices to enable monitoring of aset of demand factors and a set of Internet of Things devices deployedin proximity to a set of supply chain infrastructure facilities toenable monitoring of a set of supply factors.

In embodiments, the unified set of Internet of Things systems includes aset of workplace Internet of Things devices to enable monitoring of aset of demand factors for a set of business customers and a set ofInternet of Things devices deployed in proximity to a set of supplychain infrastructure facilities to enable monitoring of a set of supplyfactors.

In embodiments, the unified set of Internet of Things systems includes aset of Internet of Things devices to monitor a set of consumer goodsstores to enable monitoring of a set of demand factors for a set ofconsumers and a set of Internet of Things devices deployed in proximityto a set of supply chain infrastructure facilities to enable monitoringof a set of supply factors.

In embodiments, the Internet of Things systems as mentioned throughoutthis disclosure may include, for example and without limitations, camerasystems, lighting systems, motion sensing systems, weighing systems,inspection systems, machine vision systems, environmental sensorsystems, onboard sensor systems, onboard diagnostic systems,environmental control systems, sensor-enabled network switching androuting systems, RF sensing systems, magnetic sensing systems, pressuremonitoring systems, vibration monitoring systems, temperature monitoringsystems, heat flow monitoring systems, biological measurement systems,chemical measurement systems, ultrasonic monitoring systems, radiographysystems, LIDAR-based monitoring systems, access control systems,penetrating wave sensing systems, SONAR-based monitoring systems,radar-based monitoring systems, computed tomography systems, magneticresonance imaging systems, network monitoring systems, and many others.

Machine Vision Feeding Digital Twin

Referring to FIG. 34 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the platform 604 may include a machine vision system1600 and a digital twin system 1700, wherein the machine vision system1600 feeds data to the digital twin system 1700 (which may be enabled bya set of adaptive intelligent systems 614, including artificialintelligence 1160, and may be used as interfaces or components ofinterfaces 702, such as ones by which an operator may monitor twins 1700of various value chain network entities 652). The machine vision system1600 and digital twin system 1700 may operate in coordination for a setof multiple applications 630 of various types, such as a set of supplychain management applications 21004, demand management applications1502, intelligent product applications 1510 and enterprise resourcemanagement applications 1520 that monitor and/or manage a value chainnetwork and a set of value chain network entities 652.

The machine vision system 1600 and digital twin system 1700 may thusprovide, in embodiments, image-based monitoring (with automatedprocessing of image data) a wide range of facilities, devices, systems,environments, and assets, such as supply chain infrastructure facilities1560 and other value chain network entities 652 that are involved as aproduct 1510 travels from a point of origin through distribution andretail channels to an environment where it is used by a customer, aswell as representation of images, as well as extracted data from images,in a digital twin 1700. This unification may provide a number ofadvantages, including improved monitoring, improved visualization andinsight, improved visibility, and others. These may include machinevision systems 1600 and digital twin systems 1700 that are used inconnection with demand factors 1540 and supply factors 1550, so that anapplication 630 may benefit from information collected by, processed, orproduced by the machine vision system 1600 and digital twin system 1700for other applications 630 of the platform 604, and a user can developinsights about connections among the factors and control one or both ofthem with coordinated intelligence. The machine vision system 1600and/or digital twin system 1700 may be adapted over time, such as bylearning on outcomes 1040 or other operations of the adaptiveintelligent systems 614, such as to determine which elements collectedand/or processed by the machine vision system 1600 and/or digital twinsystem 1700 should be made available to which applications 630, whatelements and/or content provide the most benefit, what data should bestored or cached for immediate retrieval, what data can be discardedversus saved, what data is most beneficial to support adaptiveintelligent systems 614, and for other uses.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and for a set of applications ofat least two types from among a set of supply chain applications, a setof demand management applications, a set of intelligent productapplications and a set of enterprise resource management applicationsand having a machine vision system and a digital twin system, whereinthe machine vision system feeds data to the digital twin system.

In embodiments, the set of supply chain applications and demandmanagement applications is among any described throughout thisdisclosure or in the documents incorporated by reference herein.

In embodiments, the set of supply chain applications and demandmanagement applications includes, for example and without limitation oneor more involving inventory management, demand prediction, demandaggregation, pricing, blockchain, smart contract, positioning,placement, promotion, analytics, finance, trading, arbitrage, customeridentity management, store planning, shelf-planning, customer routeplanning, customer route analytics, commerce, ecommerce, payments,customer relationship management, sales, marketing, advertising,bidding, customer monitoring, customer process monitoring, customerrelationship monitoring, collaborative filtering, customer profiling,customer feedback, similarity analytics, customer clustering, productclustering, seasonality factor analytics, customer behavior tracking,customer behavior analytics, product design, product configuration, A/Btesting, product variation analytics, augmented reality, virtualreality, mixed reality, customer demand profiling, customer mood,emotion or affect detection, customer mood, emotion of affect analytics,business entity profiling, customer enterprise profiling, demandmatching, location-based targeting, location-based offering, point ofsale interface, point of use interface, search, advertisement, entitydiscovery, entity search, enterprise resource planning, workforcemanagement, customer digital twin, product pricing, product bundling,product and service bundling, product assortment, upsell offerconfiguration, customer feedback engagement, customer survey, or others.

In embodiments, the set of supply chain applications and demandmanagement applications may include, without limitation, one or more ofsupply chain, asset management, risk management, inventory management,blockchain, smart contract, infrastructure management, facilitymanagement, analytics, finance, trading, tax, regulatory, identitymanagement, commerce, ecommerce, payments, security, safety, vendormanagement, process management, compatibility testing, compatibilitymanagement, infrastructure testing, incident management, predictivemaintenance, logistics, monitoring, remote control, automation,self-configuration, self-healing, self-organization, logistics, reverselogistics, waste reduction, augmented reality, virtual reality, mixedreality, supply chain digital twin, vendor profiling, supplierprofiling, manufacturer profiling, logistics entity profiling,enterprise profiling, worker profiling, workforce profiling, componentsupply policy management, warehousing, distribution, fulfillment,shipping fleet management, vehicle fleet management, workforcemanagement, maritime fleet management, navigation, routing, shippingmanagement, opportunity matching, search, entity discovery, entitysearch, distribution, delivery, enterprise resource planning or otherapplications.

In embodiments, the set of supply chain applications and demandmanagement applications may include, without limitation, one or more ofasset management, risk management, inventory management, blockchain,smart contract, analytics, finance, trading, tax, regulatory, identitymanagement, commerce, ecommerce, payments, security, safety,compatibility testing, compatibility management, incident management,predictive maintenance, monitoring, remote control, automation,self-configuration, self-healing, self-organization, waste reduction,augmented reality, virtual reality, mixed reality, product design,product configuration, product updating, product maintenance, productsupport, product testing, kit configuration, kit deployment, kitsupport, kit updating, kit maintenance, kit modification, kitmanagement, product digital twin, opportunity matching, search,advertisement, entity discovery, entity search, variation, simulation,user interface, application programming interface, connectivitymanagement, natural language interface, voice/speech interface, roboticinterface, touch interface, haptic interface, vision system interface,enterprise resource planning, or other applications.

In embodiments, the set of supply chain applications and demandmanagement applications may include, without limitation, one or more ofoperations, finance, asset management, supply chain management, demandmanagement, human resource management, product management, riskmanagement, regulatory and compliance management, inventory management,infrastructure management, facilities management, analytics, trading,tax, identity management, vendor management, process management, projectmanagement, operations management, customer relationship management,workforce management, incident management, research and development,sales management, marketing management, fleet management, opportunityanalytics, decision support, strategic planning, forecasting, resourcemanagement, property management, or other applications.

In embodiments, the machine vision system includes an artificialintelligence system that is trained to recognize a type of value chainasset based on a labeled data set of images of such type of value chainassets.

In embodiments, the digital twin presents an indicator of the type ofasset based on the output of the artificial intelligence system.

In embodiments, the machine vision system includes an artificialintelligence system that is trained to recognize a type of activityinvolving a set of value chain entities based on a labeled data set ofimages of such type of activity.

In embodiments, the digital twin presents an indicator of the type ofactivity based on the output of the artificial intelligence system.

In embodiments, the machine vision system includes an artificialintelligence system that is trained to recognize a safety hazardinvolving a value chain entity based on a training data set thatincludes a set of images of value chain network activities and a set ofvalue chain network safety outcomes.

In embodiments, the digital twin presents an indicator of the hazardbased on the output of the artificial intelligence system.

In embodiments, the machine vision system includes an artificialintelligence system that is trained to predict a delay based on atraining data set that includes a set of images of value chain networkactivities and a set of value chain network timing outcomes.

In embodiments, the digital twin presents an indicator of a likelihoodof delay based on the output of the artificial intelligence system.

As noted elsewhere herein and in documents incorporated by reference,artificial intelligence (such as any of the techniques or systemsdescribed throughout this disclosure) in connection with value chainnetwork entities 652 and related processes and applications may be usedto facilitate, among other things: (a) the optimization, automationand/or control of various functions, workflows, applications, features,resource utilization and other factors, (b) recognition or diagnosis ofvarious states, entities, patterns, events, contexts, behaviors, orother elements; and/or (c) the forecasting of various states, events,contexts or other factors. As artificial intelligence improves, a largearray of domain-specific and/or general artificial intelligence systemshave become available and are likely to continue to proliferate. Asdevelopers seek solutions to domain-specific problems, such as onesrelevant to value chain entities 652 and applications 630 describedthroughout this disclosure they face challenges in selecting artificialintelligence models (such as what set of neural networks, machinelearning systems, expert systems, or the like to select) and indiscovering and selecting what inputs may enable effective and efficientuse of artificial intelligence for a given problem. As noted above,opportunity miners 1460 may assist with the discovery of opportunitiesfor increased automation and intelligence; however, once opportunitiesare discovered, selection and configuration of an artificialintelligence solution still presents a significant challenge, one thatis likely to continue to grow as artificial intelligence solutionsproliferate.

One set of solutions to these challenges is an artificial intelligencestore 3504 that is configured to enable collection, organization,recommendation and presentation of relevant sets of artificialintelligence systems based on one or more attributes of a domain and/ora domain-related problem. In embodiments, an artificial intelligencestore 3504 may include a set of interfaces to artificial intelligencesystems, such as enabling the download of relevant artificialintelligence applications, establishment of links or other connectionsto artificial intelligence systems (such as links to cloud-deployedartificial intelligence systems via APIs, ports, connectors, or otherinterfaces) and the like. The artificial intelligence store 3504 mayinclude descriptive content with respect to each of a variety ofartificial intelligence systems, such as metadata or other descriptivematerial indicating suitability of a system for solving particular typesof problems (e.g., forecasting, NLP, image recognition, patternrecognition, motion detection, route optimization, or many others)and/or for operating on domain-specific inputs, data or other entities.In embodiments, the artificial intelligence store 3504 may be organizedby category, such as domain, input types, processing types, outputtypes, computational requirements and capabilities, cost, energy usage,and other factors. In embodiments, an interface to the application store3504 may take input from a developer and/or from the platform (such asfrom an opportunity miner 1460) that indicates one or more attributes ofa problem that may be addressed through artificial intelligence and mayprovide a set of recommendations, such as via an artificial intelligenceattribute search engine, for a subset of artificial intelligencesolutions that may represent favorable candidates based on thedeveloper's domain-specific problem. Search results or recommendationsmay, in embodiments, be based at least in part on collaborativefiltering, such as by asking developers to indicate or select elementsof favorable models, as well as by clustering, such as by usingsimilarity matrices, k-means clustering, or other clustering techniquesthat associate similar developers, similar domain-specific problems,and/or similar artificial intelligence solutions. The artificialintelligence store 3504 may include e-commerce features, such asratings, reviews, links to relevant content, and mechanisms forprovisioning, licensing, delivery and payment (including allocation ofpayments to affiliates and or contributors), including ones that operateusing smart contract and/or blockchain features to automate purchasing,licensing, payment tracking, settlement of transactions, or otherfeatures.

Referring to FIG. 43 , the artificial intelligence system 1160 maydefine a machine learning model 3000 for performing analytics,simulation, decision making, and prediction making related to dataprocessing, data analysis, simulation creation, and simulation analysisof one or more of the value chain entities 652. The machine learningmodel 3000 is an algorithm and/or statistical model that performsspecific tasks without using explicit instructions, relying instead onpatterns and inference. The machine learning model 3000 builds one ormore mathematical models based on training data to make predictionsand/or decisions without being explicitly programmed to perform thespecific tasks. The machine learning model 3000 may receive inputs ofsensor data as training data, including event data 1034 and state data1140 related to one or more of the value chain entities 652. The sensordata input to the machine learning model 3000 may be used to train themachine learning model 3000 to perform the analytics, simulation,decision making, and prediction making relating to the data processing,data analysis, simulation creation, and simulation analysis of the oneor more of the value chain entities 652. The machine learning model 3000may also use input data from a user or users of the informationtechnology system. The machine learning model 3000 may include anartificial neural network, a decision tree, a support vector machine, aBayesian network, a genetic algorithm, any other suitable form ofmachine learning model, or a combination thereof. The machine learningmodel 3000 may be configured to learn through supervised learning,unsupervised learning, reinforcement learning, self-learning, featurelearning, sparse dictionary learning, anomaly detection, associationrules, a combination thereof, or any other suitable algorithm forlearning.

The artificial intelligence system 1160 may also define the digital twinsystem 1700 to create a digital replica of one or more of the valuechain entities 652. The digital replica of the one or more of the valuechain entities 652 may use substantially real-time sensor data toprovide for substantially real-time virtual representation of the valuechain entity 652 and provides for simulation of one or more possiblefuture states of the one or more value chain entities 652. The digitalreplica exists simultaneously with the one or more value chain entities652 being replicated. The digital replica provides one or moresimulations of both physical elements and properties of the one or morevalue chain entities 652 being replicated and the dynamics thereof, inembodiments, throughout the lifestyle of the one or more value chainentities 652 being replicated. The digital replica may provide ahypothetical simulation of the one or more value chain entities 652, forexample during a design phase before the one or more value chainentities are constructed or fabricated, or during or after constructionor fabrication of the one or more value chain entities by allowing forhypothetical extrapolation of sensor data to simulate a state of the oneor more value chain entities 652, such as during high stress, after aperiod of time has passed during which component wear may be an issue,during maximum throughput operation, after one or more hypothetical orplanned improvements have been made to the one or more value chainentities 652, or any other suitable hypothetical situation. In someembodiments, the machine learning model 3000 may automatically predicthypothetical situations for simulation with the digital replica, such asby predicting possible improvements to the one or more value chainentities 652, predicting when one or more components of the one or morevalue chain entities 652 may fail, and/or suggesting possibleimprovements to the one or more value chain entities 652, such aschanges to timing settings, arrangement, components, or any othersuitable change to the value chain entities 652. The digital replicaallows for simulation of the one or more value chain entities 652 duringboth design and operation phases of the one or more value chain entities652, as well as simulation of hypothetical operation conditions andconfigurations of the one or more value chain entities 652. The digitalreplica allows for invaluable analysis and simulation of the one or morevalue chain entities, by facilitating observation and measurement ofnearly any type of metric, including temperature, wear, light,vibration, etc. not only in, on, and around each component of the one ormore value chain entities 652, but in some embodiments within the one ormore value chain entities 652. In some embodiments, the machine learningmodel 3000 may process the sensor data including the event data 1034 andthe state data 1140 to define simulation data for use by the digitaltwin system 1700. The machine learning model 3000 may, for example,receive state data 1140 and event data 1034 related to a particularvalue chain entity 652 of the plurality of value chain entities 652 andperform a series of operations on the state data 1140 and the event data1034 to format the state data 1140 and the event data 1034 into a formatsuitable for use by the digital twin system 1700 in creation of adigital replica of the value chain entity 652. For example, one or morevalue chain entities 652 may include a robot configured to augmentproducts on an adjacent assembly line. The machine learning model 3000may collect data from one or more sensors positioned on, near, in,and/or around the robot. The machine learning model 3000 may performoperations on the sensor data to process the sensor data into simulationdata and output the simulation data to the digital twin system 1700. Thedigital twin simulation 1700 may use the simulation data to create oneor more digital replicas of the robot, the simulation including forexample metrics including temperature, wear, speed, rotation, andvibration of the robot and components thereof. The simulation may be asubstantially real-time simulation, allowing for a human user of theinformation technology to view the simulation of the robot, metricsrelated thereto, and metrics related to components thereof, insubstantially real time. The simulation may be a predictive orhypothetical situation, allowing for a human user of the informationtechnology to view a predictive or hypothetical simulation of the robot,metrics related thereto, and metrics related to components thereof.

In some embodiments, the machine learning model 3000 and the digitaltwin system 1700 may process sensor data and create a digital replica ofa set of value chain entities of the plurality of value chain entities652 to facilitate design, real-time simulation, predictive simulation,and/or hypothetical simulation of a related group of value chainentities. The digital replica of the set of value chain entities may usesubstantially real-time sensor data to provide for substantiallyreal-time virtual representation of the set of value chain entities andprovide for simulation of one or more possible future states of the setof value chain entities. The digital replica exists simultaneously withthe set of value chain entities being replicated. The digital replicaprovides one or more simulations of both physical elements andproperties of the set of value chain entities being replicated and thedynamics thereof, in embodiments throughout the lifestyle of the set ofvalue chain entities being replicated. The one or more simulations mayinclude a visual simulation, such as a wire-frame virtual representationof the one or more value chain entities 652 that may be viewable on amonitor, using an augmented reality (AR) apparatus, or using a virtualreality (VR) apparatus. The visual simulation may be able to bemanipulated by a human user of the information technology system, suchas zooming or highlighting components of the simulation and/or providingan exploded view of the one or more value chain entities 652. Thedigital replica may provide a hypothetical simulation of the set ofvalue chain entities, for example during a design phase before the oneor more value chain entities are constructed or fabricated, or during orafter construction or fabrication of the one or more value chainentities by allowing for hypothetical extrapolation of sensor data tosimulate a state of the set of value chain entities, such as during highstress, after a period of time has passed during which component wearmay be an issue, during maximum throughput operation, after one or morehypothetical or planned improvements have been made to the set of valuechain entities, or any other suitable hypothetical situation. In someembodiments, the machine learning model 3000 may automatically predicthypothetical situations for simulation with the digital replica, such asby predicting possible improvements to the set of value chain entities,predicting when one or more components of the set of value chainentities may fail, and/or suggesting possible improvements to the set ofvalue chain entities, such as changes to timing settings, arrangement,components, or any other suitable change to the value chain entities652. The digital replica allows for simulation of the set of value chainentities during both design and operation phases of the set of valuechain entities, as well as simulation of hypothetical operationconditions and configurations of the set of value chain entities. Thedigital replica allows for invaluable analysis and simulation of the oneor more value chain entities, by facilitating observation andmeasurement of nearly any type of metric, including temperature, wear,light, vibration, etc. not only in, on, and around each component of theset of value chain entities, but in some embodiments within the set ofvalue chain entities. In some embodiments, the machine learning model3000 may process the sensor data including the event data 1034 and thestate data 1140 to define simulation data for use by the digital twinsystem 1700. The machine learning model 3000 may, for example, receivestate data 1140 and event data 1034 related to a particular value chainentity 652 of the plurality of value chain entities 652 and perform aseries of operations on the state data 1140 and the event data 1034 toformat the state data 1140 and the event data 1034 into a formatsuitable for use by the digital twin system 1700 in the creation of adigital replica of the set of value chain entities. For example, a setof value chain entities may include a die machine configured to placeproducts on a conveyor belt, the conveyor belt on which the die machineis configured to place the products, and a plurality of robotsconfigured to add parts to the products as they move along the assemblyline. The machine learning model 3000 may collect data from one or moresensors positioned on, near, in, and/or around each of the die machines,the conveyor belt, and the plurality of robots. The machine learningmodel 3000 may perform operations on the sensor data to process thesensor data into simulation data and output the simulation data to thedigital twin system 1700. The digital twin simulation 1700 may use thesimulation data to create one or more digital replicas of the diemachine, the conveyor belt, and the plurality of robots, the simulationincluding for example metrics including temperature, wear, speed,rotation, and vibration of the die machine, the conveyor belt, and theplurality of robots and components thereof. The simulation may be asubstantially real-time simulation, allowing for a human user of theinformation technology to view the simulation of the die machine, theconveyor belt, and the plurality of robots, metrics related thereto, andmetrics related to components thereof, in substantially real time. Thesimulation may be a predictive or hypothetical situation, allowing for ahuman user of the information technology to view a predictive orhypothetical simulation of the die machine, the conveyor belt, and theplurality of robots, metrics related thereto, and metrics related tocomponents thereof.

In some embodiments, the machine learning model 3000 may prioritizecollection of sensor data for use in digital replica simulations of oneor more of the value chain entities 652. The machine learning model 3000may use sensor data and user inputs to train, thereby learning whichtypes of sensor data are most effective for creation of digitalreplicate simulations of one or more of the value chain entities 652.For example, the machine learning model 3000 may find that a particularvalue chain entity 652 has dynamic properties such as component wear andthroughput affected by temperature, humidity, and load. The machinelearning model 3000 may, through machine learning, prioritize collectionof sensor data related to temperature, humidity, and load, and mayprioritize processing sensor data of the prioritized type intosimulation data for output to the digital twin system 1700. In someembodiments, the machine learning model 3000 may suggest to a user ofthe information technology system that more and/or different sensors ofthe prioritized type be implemented in the information technology andvalue chain system near and around the value chain entity 652 beingsimulation such that more and/or better data of the prioritized type maybe used in simulation of the value chain entity 652 via the digitalreplica thereof.

In some embodiments, the machine learning model 3000 may be configuredto learn to determine which types of sensor data are to be processedinto simulation data for transmission to the digital twin system 1700based on one or both of a modeling goal and a quality or type of sensordata. A modeling goal may be an objective set by a user of theinformation technology system or may be predicted or learned by themachine learning model 3000. Examples of modeling goals include creatinga digital replica capable of showing dynamics of throughput on anassembly line, which may include collection, simulation, and modelingof, e.g., thermal, electrical power, component wear, and other metricsof a conveyor belt, an assembly machine, one or more products, and othercomponents of the value chain. The machine learning model 3000 may beconfigured to learn to determine which types of sensor data arenecessary to be processed into simulation data for transmission to thedigital twin system 1700 to achieve such a model. In some embodiments,the machine learning model 3000 may analyze which types of sensor dataare being collected, the quality and quantity of the sensor data beingcollected, and what the sensor data being collected represents, and maymake decisions, predictions, analyses, and/or determinations related towhich types of sensor data are and/or are not relevant to achieving themodeling goal and may make decisions, predictions, analyses, and/ordeterminations to prioritize, improve, and/or achieve the quality andquantity of sensor data being processed into simulation data for use bythe digital twin system 1700 in achieving the modeling goal.

In some embodiments, a user of the information technology system mayinput a modeling goal into the machine learning model 3000. The machinelearning model 3000 may learn to analyze training data to outputsuggestions to the user of the information technology system regardingwhich types of sensor data are most relevant to achieving the modelinggoal, such as one or more types of sensors positioned in, on, or near avalue chain entity or a plurality of value chain entities that isrelevant to the achievement of the modeling goal is and/or are notsufficient for achieving the modeling goal, and how a differentconfiguration of the types of sensors, such as by adding, removing, orrepositioning sensors, may better facilitate achievement of the modelinggoal by the machine learning model 3000 and the digital twin system1700. In some embodiments, the machine learning model 3000 mayautomatically increase or decrease collection rates, processing,storage, sampling rates, bandwidth allocation, bitrates, and otherattributes of sensor data collection to achieve or better achieve themodeling goal. In some embodiments, the machine learning model 3000 maymake suggestions or predictions to a user of the information technologysystem related to increasing or decreasing collection rates, processing,storage, sampling rates, bandwidth allocation, bitrates, and otherattributes of sensor data collection to achieve or better achieve themodeling goal. In some embodiments, the machine learning model 3000 mayuse sensor data, simulation data, previous, current, and/or futuredigital replica simulations of one or more value chain entities 652 ofthe plurality of value chain entities 652 to automatically create and/orpropose modeling goals. In some embodiments, modeling goalsautomatically created by the machine learning model 3000 may beautomatically implemented by the machine learning model 3000. In someembodiments, modeling goals automatically created by the machinelearning model 3000 may be proposed to a user of the informationtechnology system, and implemented only after acceptance and/or partialacceptance by the user, such as after modifications are made to theproposed modeling goal by the user.

In some embodiments, the user may input the one or more modeling goals,for example, by inputting one or more modeling commands to theinformation technology system. The one or more modeling commands mayinclude, for example, a command for the machine learning model 3000 andthe digital twin system 1700 to create a digital replica simulation ofone value chain entity 652 or a set of value chain entities of theplurality of 652, may include a command for the digital replicasimulation to be one or more of a real-time simulation, and ahypothetical simulation. The modeling command may also include, forexample, parameters for what types of sensor data should be used,sampling rates for the sensor data, and other parameters for the sensordata used in the one or more digital replica simulations. In someembodiments, the machine learning model 3000 may be configured topredict modeling commands, such as by using previous modeling commandsas training data. The machine learning model 3000 may propose predictedmodeling commands to a user of the information technology system, forexample, to facilitate simulation of one or more of the value chainentities 652 that may be useful for the management of the value chainentities 652 and/or to allow the user to easily identify potentialissues with or possible improvements to the value chain entities 652.

In some embodiments, the machine learning model 3000 may be configuredto evaluate a set of hypothetical simulations of one or more of thevalue chain entities 652. The set of hypothetical simulations may becreated by the machine learning model 3000 and the digital twin system1700 as a result of one or more modeling commands, as a result of one ormore modeling goals, one or more modeling commands, by prediction by themachine learning model 3000, or a combination thereof. The machinelearning model 3000 may evaluate the set of hypothetical simulationsbased on one or more metrics defined by the user, one or more metricsdefined by the machine learning model 3000, or a combination thereof. Insome embodiments, the machine learning model 3000 may evaluate each ofthe hypothetical simulations of the set of hypothetical simulationsindependently of one another. In some embodiments, the machine learningmodel 3000 may evaluate one or more of the hypothetical simulations ofthe set of hypothetical simulations in relation to one another, forexample by ranking the hypothetical simulations or creating tiers of thehypothetical simulations based on one or more metrics.

In some embodiments, the machine learning model 3000 may include one ormore model interpretability systems to facilitate human understanding ofoutputs of the machine learning model 3000, as well as information andinsight related to cognition and processes of the machine learning model3000, i.e., the one or more model interpretability systems allow forhuman understanding of not only “what” the machine learning model 3000is outputting, but also “why” the machine learning model 3000 isoutputting the outputs thereof, and what process led to the 3000formulating the outputs. The one or more model interpretability systemsmay also be used by a human user to improve and guide training of themachine learning model 3000, to help debug the machine learning model3000, to help recognize bias in the machine learning model 3000. The oneor more model interpretability systems may include one or more of linearregression, logistic regression, a generalized linear model (GLM), ageneralized additive model (GAM), a decision tree, a decision rule,RuleFit, Naive Bayes Classifier, a K-nearest neighbors algorithm, apartial dependence plot, individual conditional expectation (ICE), anaccumulated local effects (ALE) plot, feature interaction, permutationfeature importance, a global surrogate model, a local surrogate (LIME)model, scoped rules, i.e., anchors, Shapley values, Shapley additiveexplanations (SHAP), feature visualization, network dissection, or anyother suitable machine learning interpretability implementation. In someembodiments, the one or more model interpretability systems may includea model dataset visualization system. The model dataset visualizationsystem is configured to automatically provide to a human user of theinformation technology system visual analysis related to distribution ofvalues of the sensor data, the simulation data, and data nodes of themachine learning model 3000.

In some embodiments, the machine learning model 3000 may include and/orimplement an embedded model interpretability system, such as a Bayesiancase model (BCM) or glass box. The Bayesian case model uses Bayesiancase-based reasoning, prototype classification, and clustering tofacilitate human understanding of data such as the sensor data, thesimulation data, and data nodes of the machine learning model 3000. Insome embodiments, the model interpretability system may include and/orimplement a glass box interpretability method, such as a Gaussianprocess, to facilitate human understanding of data such as the sensordata, the simulation data, and data nodes of the machine learning model3000.

In some embodiments, the machine learning model 3000 may include and/orimplement testing with concept activation vectors (TCAV). The TCAVallows the machine learning model 3000 to learn human-interpretableconcepts, such as “running,” “not running,” “powered,” “not powered,”“robot,” “human,” “truck,” or “ship” from examples by a processincluding defining the concept, determining concept activation vectors,and calculating directional derivatives. By learning human-interpretableconcepts, objects, states, etc., TCAV may allow the machine learningmodel 3000 to output useful information related to the value chainentities 652 and data collected therefrom in a format that is readilyunderstood by a human user of the information technology system.

In some embodiments, the machine learning model 3000 may be and/orinclude an artificial neural network, e.g., a connectionist systemconfigured to “learn” to perform tasks by considering examples andwithout being explicitly programmed with task-specific rules. Themachine learning model 3000 may be based on a collection of connectedunits and/or nodes that may act like artificial neurons that may in someways emulate neurons in a biological brain. The units and/or nodes mayeach have one or more connections to other units and/or nodes. The unitsand/or nodes may be configured to transmit information, e.g., one ormore signals, to other units and/or nodes, process signals received fromother units and/or nodes, and forward processed signals to other unitsand/or nodes. One or more of the units and/or nodes and connectionstherebetween may have one or more numerical “weights” assigned. Theassigned weights may be configured to facilitate learning, i.e.,training, of the machine learning model 3000. The weights assignedweights may increase and/or decrease one or more signals between one ormore units and/or nodes, and in some embodiments may have one or morethresholds associated with one or more of the weights. The one or morethresholds may be configured such that a signal is only sent between oneor more units and/or nodes, if a signal and/or aggregate signal crossesthe threshold. In some embodiments, the units and/or nodes may beassigned to a plurality of layers, each of the layers having one or bothof inputs and outputs. A first layer may be configured to receivetraining data, transform at least a portion of the training data, andtransmit signals related to the training data and transformation thereofto a second layer. A final layer may be configured to output anestimate, conclusion, product, or other consequence of processing of oneor more inputs by the machine learning model 3000. Each of the layersmay perform one or more types of transformations, and one or moresignals may pass through one or more of the layers one or more times. Insome embodiments, the machine learning model 3000 may employ deeplearning and being at least partially modeled and/or configured as adeep neural network, a deep belief network, a recurrent neural network,and/or a convolutional neural network, such as by being configured toinclude one or more hidden layers.

In some embodiments, the machine learning model 3000 may be and/orinclude a decision tree, e.g., a tree-based predictive model configuredto identify one or more observations and determine one or moreconclusions based on an input. The observations may be modeled as one ormore “branches” of the decision tree, and the conclusions may be modeledas one or more “leaves” of the decision tree. In some embodiments, thedecision tree may be a classification tree. the classification tree mayinclude one or more leaves representing one or more class labels, andone or more branches representing one or more conjunctions of featuresconfigured to lead to the class labels. In some embodiments, thedecision tree may be a regression tree. The regression tree may beconfigured such that one or more target variables may take continuousvalues.

In some embodiments, the machine learning model 3000 may be and/orinclude a support vector machine, e.g., a set of related supervisedlearning methods configured for use in one or both of classification andregression-based modeling of data. The support vector machine may beconfigured to predict whether a new example falls into one or morecategories, the one or more categories being configured during trainingof the support vector machine.

In some embodiments, the machine learning model 3000 may be configuredto perform regression analysis to determine and/or estimate arelationship between one or more inputs and one or more features of theone or more inputs. Regression analysis may include linear regression,wherein the machine learning model 3000 may calculate a single line tobest fit input data according to one or more mathematical criteria.

In embodiments, inputs to the machine learning model 3000 (such as aregression model, Bayesian network, supervised model, or other type ofmodel) may be tested, such as by using a set of testing data that isindependent from the data set used for the creation and/or training ofthe machine learning model, such as to test the impact of various inputsto the accuracy of the model 3000. For example, inputs to the regressionmodel may be removed, including single inputs, pairs of inputs,triplets, and the like, to determine whether the absence of inputscreates a material degradation of the success of the model 3000. Thismay assist with recognition of inputs that are in fact correlated (e.g.,are linear combinations of the same underlying data), that areoverlapping, or the like. Comparison of model success may help selectamong alternative input data sets that provide similar information, suchas to identify the inputs (among several similar ones) that generate theleast “noise” in the model, that provide the most impact on modeleffectiveness for the lowest cost, or the like. Thus, input variationand testing of the impact of input variation on model effectiveness maybe used to prune or enhance model performance for any of the machinelearning systems described throughout this disclosure.

In some embodiments, the machine learning model 3000 may be and/orinclude a Bayesian network. The Bayesian network may be a probabilisticgraphical model configured to represent a set of random variables andconditional independence of the set of random variables. The Bayesiannetwork may be configured to represent the random variables andconditional independence via a directed acyclic graph. The Bayesiannetwork may include one or both of a dynamic Bayesian network and aninfluence diagram.

In some embodiments, the machine learning model 3000 may be defined viasupervised learning, i.e., one or more algorithms configured to build amathematical model of a set of training data containing one or moreinputs and desired outputs. The training data may consist of a set oftraining examples, each of the training examples having one or moreinputs and desired outputs, i.e., a supervisory signal. Each of thetraining examples may be represented in the machine learning model 3000by an array and/or a vector, i.e., a feature vector. The training datamay be represented in the machine learning model 3000 by a matrix. Themachine learning model 3000 may learn one or more functions viaiterative optimization of an objective function, thereby learning topredict an output associated with new inputs. Once optimized, theobjective function may provide the machine learning model 3000 with theability to accurately determine an output for inputs other than inputsincluded in the training data. In some embodiments, the machine learningmodel 3000 may be defined via one or more supervised learning algorithmssuch as active learning, statistical classification, regressionanalysis, and similarity learning. Active learning may includeinteractively querying, by the machine learning model AILD 102T, a userand/or an information source to label new data points with desiredoutputs. Statistical classification may include identifying, by themachine learning model 3000, to which a set of subcategories, i.e.,subpopulations, a new observation belongs based on a training set ofdata containing observations having known categories. Regressionanalysis may include estimating, by the machine learning model 3000relationships between a dependent variable, i.e., an outcome variable,and one or more independent variables, i.e., predictors, covariates,and/or features. Similarity learning may include learning, by themachine learning model 3000, from examples using a similarity function,the similarity function being designed to measure how similar or relatedtwo objects are.

In some embodiments, the machine learning model 3000 may be defined viaunsupervised learning, i.e., one or more algorithms configured to builda mathematical model of a set of data containing only inputs by findingstructure in the data such as grouping or clustering of data points. Insome embodiments, the machine learning model 3000 may learn from testdata, i.e., training data, that has not been labeled, classified, orcategorized. The unsupervised learning algorithm may includeidentifying, by the machine learning model 3000, commonalities in thetraining data and learning by reacting based on the presence or absenceof the identified commonalities in new pieces of data. In someembodiments, the machine learning model 3000 may generate one or moreprobability density functions. In some embodiments, the machine learningmodel 3000 may learn by performing cluster analysis, such as byassigning a set of observations into subsets, i.e., clusters, accordingto one or more predesignated criteria, such as according to a similaritymetric of which internal compactness, separation, estimated density,and/or graph connectivity are factors.

In some embodiments, the machine learning model 3000 may be defined viasemi-supervised learning, i.e., one or more algorithms using trainingdata wherein some training examples may be missing training labels. Thesemi-supervised learning may be weakly supervised learning, wherein thetraining labels may be noisy, limited, and/or imprecise. The noisy,limited, and/or imprecise training labels may be cheaper and/or lesslabor intensive to produce, thus allowing the machine learning model3000 to train on a larger set of training data for less cost and/orlabor.

In some embodiments, the machine learning model 3000 may be defined viareinforcement learning, such as one or more algorithms using dynamicprogramming techniques such that the machine learning model 3000 maytrain by taking actions in an environment in order to maximize acumulative reward. In some embodiments, the training data is representedas a Markov Decision Process.

In some embodiments, the machine learning model 3000 may be defined viaself-learning, wherein the machine learning model 3000 is configured totrain using training data with no external rewards and no externalteaching, such as by employing a Crossbar Adaptive Array (CAA). The CAAmay compute decisions about actions and/or emotions about consequencesituations in a crossbar fashion, thereby driving teaching of themachine learning model 3000 by interactions between cognition andemotion.

In some embodiments, the machine learning model 3000 may be defined viafeature learning, i.e., one or more algorithms designed to discoverincreasingly accurate and/or apt representations of one or more inputsprovided during training, e.g., training data. Feature learning mayinclude training via principal component analysis and/or clusteranalysis. Feature learning algorithms may include attempting, by themachine learning model 3000, to preserve input training data while alsotransforming the input training data such that the transformed inputtraining data is useful. In some embodiments, the machine learning model3000 may be configured to transform the input training data prior toperforming one or more classifications and/or predictions of the inputtraining data. Thus, the machine learning model 3000 may be configuredto reconstruct input training data from one or more unknowndata-generating distributions without necessarily conforming toimplausible configurations of the input training data according to thedistributions. In some embodiments, the feature learning algorithm maybe performed by the machine learning model 3000 in a supervised,unsupervised, or semi-supervised manner.

In some embodiments, the machine learning model 3000 may be defined viaanomaly detection, i.e., by identifying rare and/or outlier instances ofone or more items, events and/or observations. The rare and/or outlierinstances may be identified by the instances differing significantlyfrom patterns and/or properties of a majority of the training data.Unsupervised anomaly detection may include detecting of anomalies, bythe machine learning model 3000, in an unlabeled training data set underan assumption that a majority of the training data is “normal.”Supervised anomaly detection may include training on a data set whereinat least a portion of the training data has been labeled as “normal”and/or “abnormal.”

In some embodiments, the machine learning model 3000 may be defined viarobot learning. Robot learning may include generation, by the machinelearning model 3000, of one or more curricula, the curricula beingsequences of learning experiences, and cumulatively acquiring new skillsvia exploration guided by the machine learning model 3000 and socialinteraction with humans by the machine learning model 3000. Acquisitionof new skills may be facilitated by one or more guidance mechanisms suchas active learning, maturation, motor synergies, and/or imitation.

In some embodiments, the machine learning model 3000 can be defined viaassociation rule learning. Association rule learning may includediscovering relationships, by the machine learning model 3000, betweenvariables in databases, in order to identify strong rules using somemeasure of “interestingness.” Association rule learning may includeidentifying, learning, and/or evolving rules to store, manipulate and/orapply knowledge. The machine learning model 3000 may be configured tolearn by identifying and/or utilizing a set of relational rules, therelational rules collectively representing knowledge captured by themachine learning model 3000. Association rule learning may include oneor more of learning classifier systems, inductive logic programming, andartificial immune systems. Learning classifier systems are algorithmsthat may combine a discovery component, such as one or more geneticalgorithms, with a learning component, such as one or more algorithmsfor supervised learning, reinforcement learning, or unsupervisedlearning. Inductive logic programming may include rule-learning, by themachine learning model 3000, using logic programming to represent one ormore of input examples, background knowledge, and hypothesis determinedby the machine learning model 3000 during training. The machine learningmodel 3000 may be configured to derive a hypothesized logic programentailing all positive examples given an encoding of known backgroundknowledge and a set of examples represented as a logical database offacts.

In embodiments, another set of solutions, which may be deployed alone orin connection with other elements of the platform, including theartificial intelligence store 3504, may include a set of functionalimaging capabilities 3502, which may comprise monitoring systems 640 andin some cases physical process observation systems 1510 and/or softwareinteraction observation systems 1500, such as for monitoring variousvalue chain entities 652. Functional imaging systems 3502 may, inembodiments, provide considerable insight into the types of artificialintelligence that are likely to be most effective in solving particulartypes of problems most effectively. As noted elsewhere in thisdisclosure and in the documents incorporated by reference herein,computational and networking systems, as they grow in scale, complexityand interconnections, manifest problems of information overload, noise,network congestion, energy waste, and many others. As the Internet ofThings grows to hundreds of billions of devices, and virtually countlesspotential interconnections, optimization becomes exceedingly difficult.One source for insight is the human brain, which faces similarchallenges and has evolved, over millennia, reasonable solutions to awide range of very difficult optimization problems. The human brainoperates with a massive neural network organized into interconnectedmodular systems, each of which has a degree of adaptation to solveparticular problems, from regulation of biological systems andmaintenance of homeostasis, to detection of a wide range of static anddynamic patterns, to recognition of threats and opportunities, amongmany others. Functional imaging 3502, such as functional magneticresonance imaging (fMRI), electroencephalogram (EEG), computedtomography (CT) and other brain imaging systems have improved to thepoint that patterns of brain activity can be recognized in real time andtemporally associated with other information, such behaviors, stimulusinformation, environmental condition data, gestures, eye movements, andother information, such that via functional imaging, either alone or incombination with other information collected by monitoring systems 808,the platform may determine and classify what brain modules, operations,systems, and/or functions are employed during the undertaking of a setof tasks or activities, such as ones involving software interaction1500, physical process observations 1510, or a combination thereof. Thisclassification may assist in selection and/or configuration of a set ofartificial intelligence solutions, such as from an artificialintelligence store 3504, that includes a similar set of capabilitiesand/or functions to the set of modules and functions of the human brainwhen undertaking an activity, such as for the initial configuration of arobotic process automation (RPA) system 1442 that automates a taskperformed by an expert human. Thus, the platform may include a systemthat takes input from a functional imaging system to configure,optionally automatically based on matching of attributes between one ormore biological systems, such as brain systems, and one or moreartificial intelligence systems, a set of artificial intelligencecapabilities for a robotic process automation system. Selection andconfiguration may further comprise selection of inputs to roboticprocess automation and/or artificial intelligence that are configured atleast in part based on functional imaging of the brain while workersundertake tasks, such as selection of visual inputs (such as images fromcameras) where vision systems of the brain are highly activated,selection of acoustic inputs where auditory systems of the brain arehighly activated, selection of chemical inputs (such as chemicalsensors) where olfactory systems of the brain are highly activated, orthe like. Thus, a biologically aware robotic process automation systemmay be improved by having initial configuration, or iterativeimprovement, be guided, either automatically or under developer control,by imaging-derived information collected as workers perform expert tasksthat may benefit from automation.

Referring to FIG. 27 , additional details of an embodiment of theplatform 604 are provided, in particular relating to elements of theadaptive intelligence layer 614 that facilitate improved edgeintelligence, including the adaptive edge compute management system 1400and the edge intelligence system 1420. These elements provide a set ofsystems that adaptively manage “edge” computation, storage andprocessing, such as by varying storage locations for data and processinglocations (e.g., optimized by AI) between on-device storage, localsystems, in the network and in the cloud. These elements enablefacilitation of a dynamic definition by a user, such as a developer,operator, or host of the platform 102, of what constitutes the “edge”for purposes of a given application. For example, for environments wheredata connections are slow or unreliable (such as where a facility doesnot have good access to cellular networks (such as due to remoteness ofsome environments (such as in geographies with poor cellular networkinfrastructure), shielding or interference (such as where density ofnetwork-using systems, thick metals hulls of container ships, thickmetal container walls, underwater or underground location, or presenceof large metal objects (such as vaults, hulls, containers and the like)interferes with networking performance), and/or congestion (such aswhere there are many devices seeking access to limited networkingfacilities), edge computing capabilities can be defined and deployed tooperate on the local area network of an environment, in peer-to-peernetworks of devices, or on computing capabilities of local value chainentities 652. For example, in an environment with a limited set ofcomputational and/or networking resources, tasks may be intelligentlyload balanced based on a current context (e.g., network availability,latency, congestion, and the like) and, in an example, one type of datamay be prioritized for processing, or one workflow prioritized overanother workflow, and the like. Where strong data connections areavailable (such as where good backhaul facilities exist), edge computingcapabilities can be disposed in the network, such as for cachingfrequently used data at locations that improve input/output performance,reduce latency, or the like. Thus, adaptive definition and specificationof where edge computing operations are enabled, under control of adeveloper or operator, or optionally determined automatically, such asby an expert system or automation system, such as based on detectednetwork conditions for an environment, for a financial entity 652, orfor a network as a whole.

In embodiments, edge intelligence 1420 enables adaptation of edgecomputation (including where computation occurs within various availablenetworking resources, how networking occurs (such as by protocolselection), where data storage occurs, and the like) that ismulti-application aware, such as accounting for QoS, latencyrequirements, congestion, and cost as understood and prioritized basedon awareness of the requirements, the prioritization, and the value(including ROI, yield, and cost information, such as costs of failure)of edge computation capabilities across more than one application,including any combinations and subsets of the applications 630 describedherein or in the documents incorporated herein by reference.

Referring to FIG. 35 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the platform 604 may include a unified set of adaptiveedge computing and other edge intelligence systems 1420 that providecoordinated edge computation and other edge intelligence 1420capabilities for a set of multiple applications 630 of various types,such as a set of supply chain management applications 21004, demandmanagement applications 1502, intelligent product applications 1510 andenterprise resource management applications 1520 that monitor and/ormanage a value chain network and a set of value chain network entities652. In embodiments, edge intelligence capabilities of the systems andmethods described herein may include, but are not limited to, on-premiseedge devices and resources, such as local area network resources, andnetwork edge devices, such as those deployed at the edge of a cellularnetwork or within a peripheral data center, both of which may deployedge intelligence, as described herein, to, for example, carry outintelligent processing tasks at these edge locations before transferringdata or other matter, to the primary or core cellular network command orcentral data center.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a unified set of adaptive edgecomputing systems that provide coordinated edge computation for a set ofapplications of at least two types from among a set of demand managementapplications, a set of supply chain applications, a set of intelligentproduct applications and a set of enterprise resource managementapplications for a category of goods.

The adaptive edge computing and other edge intelligence systems 1420 maythus provide, in embodiments, intelligence for monitoring, managing,controlling, or otherwise handling a wide range of facilities, devices,systems, environments, and assets, such as supply chain infrastructurefacilities 1560 and other value chain network entities 652 that areinvolved as a product 1510 travels from a point of origin throughdistribution and retail channels to an environment where it is used by acustomer. This unification may provide a number of advantages, includingimproved monitoring, improved remote control, improved autonomy,improved prediction, improved classification, improved visualization andinsight, improved visibility, and others. These may include adaptiveedge computing and other edge intelligence systems 1420 that are used inconnection with demand factors 1540 and supply factors 1550, so that anapplication 630 may benefit from information collected by, processed by,or produced by the adaptive edge computing and other edge intelligencesystems 1420 for other applications 630 of the platform 604, and a usercan develop insights about connections among the factors and control oneor both of them with coordinated intelligence. For example, coordinatedintelligence may include, but is not limited to, analytics andprocessing for monitoring data streams, as described herein, for thepurposes of classification, prediction or some other type of analyticmodeling. Such coordinated intelligence methods and systems may beapplied in an automated manner in which differing combinations ofintelligence assets are applied. As an example, within an industrialenvironment the coordinated intelligence system may monitor signalscoming from machinery deployed in the environment. The coordinatedintelligence system may classify, predict or perform some otherintelligent analytics, in combination, for the purpose of, for example,determining a state of a machine, such as a machine in a deterioratedstate, in an at-risk state, or some other state. The determination of astate may cause a control system to alter a control regime, for example,slowing or shutting down a machine that is in a deteriorating state. Inembodiments, the coordinated intelligence system may coordinate acrossmultiple entities of a value chain, supply chain and the like. Forexample, the monitoring of the deteriorating machine in the industrialenvironment may simultaneously occur with analytics related to partssuppliers and availability, product supply and inventory predictions, orsome other coordinated intelligence operation. The adaptive edgecomputing and other edge intelligence systems 1420 may be adapted overtime, such as by learning on outcomes 1040 or other operations of theother adaptive intelligent systems 614, such as to determine whichelements collected and/or processed by the adaptive edge computing andother edge intelligence systems 1420 should be made available to whichapplications 630, what elements and/or content provide the most benefit,what data should be stored or cached for immediate retrieval, what datacan be discarded versus saved, what data is most beneficial to supportadaptive intelligent systems 614, and for other uses.

Referring to FIG. 36 , in embodiments, the unified set of adaptive edgecomputing systems that provide coordinated edge computation include awide range of systems, such as classification systems 1610 (such asimage classification systems, object type recognition systems, andothers), video processing systems 1612 (such as video compressionsystems), signal processing systems 1614 (such as analog-to-digitaltransformation systems, digital-to-analog transformation systems, RFfiltering systems, analog signal processing systems, multiplexingsystems, statistical signal processing systems, signal filteringsystems, natural language processing systems, sound processing systems,ultrasound processing systems, and many others), data processing systems1630 (such as data filtering systems, data integration systems, dataextraction systems, data loading systems, data transformation systems,point cloud processing systems, data normalization systems, datacleansing system, data deduplication systems, graph-based data storagesystems, object-oriented data storage systems, and others), predictivesystems 1620 (such as motion prediction systems, output predictionsystems, activity prediction systems, fault prediction systems, failureprediction systems, accident prediction systems, event predictionssystems, event prediction systems, and many others), configurationsystems 1630 (such as protocol selection systems, storage configurationsystems, peer-to-peer network configuration systems, power managementsystems, self-configuration systems, self-healing systems, handshakenegotiation systems, and others), artificial intelligence systems 1160(such as clustering systems, variation systems, machine learningsystems, expert systems, rule-based systems, deep learning systems, andmany others), system management and control systems 1640 (such asautonomous control systems, robotic control systems, RF spectrummanagement systems, network resource management systems, storagemanagement systems, data management systems, and others), roboticprocess automation systems, analytic and modeling systems 1650 (such asdata visualization systems, clustering systems, similarity analysissystems, random forest systems, physical modeling systems, interactionmodeling systems, simulation systems, and many others), entity discoverysystems, security systems 1670 (such as cybersecurity systems, biometricsystems, intrusion detection systems, firewall systems, and others),rules engine systems, workflow automation systems, opportunity discoverysystems, testing and diagnostic systems 1660, software image propagationsystems, virtualization systems, digital twin systems, Internet ofThings monitoring systems, routing systems, switching systems, indoorlocation systems, geolocation systems, and others.

In embodiments, the interface is a user interface for a command centerdashboard by which an enterprise orchestrates a set of value chainentities related to a type of product.

In embodiments, the interface is a user interface of a local managementsystem located in an environment that hosts a set of value chainentities.

In embodiments, the local management system user interface facilitatesconfiguration of a set of network connections for the adaptive edgecomputing systems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of data storage resources for the adaptive edgecomputing systems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of data integration capabilities for the adaptiveedge computing systems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of machine learning input resources for theadaptive edge computing systems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of power resources that support the adaptive edgecomputing systems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of workflows that are managed by the adaptiveedge computing systems.

In embodiments, the interface is a user interface of a mobile computingdevice that has a network connection to the adaptive edge computingsystems.

In embodiments, the interface is an application programming interface.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and acloud-based artificial intelligence system.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and areal-time operating system of a cloud data management platform.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and acomputational facility of a cloud data management platform.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and a setof environmental sensors that collect data about an environment thathosts a set of value chain network entities.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and a setof sensors that collect data about a product.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and a setof sensors that collect data published by an intelligent product.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive edge computing systems and a setof sensors that collect data published by a set of Internet of Thingssystems that are disposed in an environment that hosts a set of valuechain network entities.

In embodiments, the set of demand management applications, supply chainapplications, intelligent product applications and enterprise resourcemanagement applications may include, for example, any of theapplications mentioned throughout this disclosure or in the documentsincorporated by reference herein.

Unified Adaptive Intelligence

Referring to FIG. 37 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

In embodiments, the VCNP 102 may include a unified set of adaptiveintelligent systems 614 that provide coordinated intelligence for a setof various applications, such as demand management applications 1502, aset of supply chain applications 1500, a set of intelligent productapplications 1510, a set of enterprise resource management applications1520 and a set of asset management applications 1530 for a category ofgoods.

In embodiments, the unified set of adaptive intelligence systems includea wide variety of systems described throughout this disclosure and inthe documents incorporated herein by reference, such as, withoutlimitation, the edge intelligence systems 1420, classification systems1610, data processing systems 1612, signal processing systems 1614,artificial intelligence systems 1160, prediction systems 1620,configuration systems 1630, control systems 1640, analytic systems 1650,testing/diagnostic systems 1660, security systems 1670 and othersystems, whether used for edge intelligence or for intelligence within anetwork, within an application, or in the cloud, as well as to servevarious layers of the platform 604. These include neural networks, deeplearning systems, model-based systems, expert systems, machine learningsystems, rule-based systems, opportunity miners, robotic processautomation systems, data transformation systems, data extractionsystems, data loading systems, genetic programming systems, imageclassification systems, video compression systems, analog-to-digitaltransformation systems, digital-to-analog transformation systems, signalanalysis systems, RF filtering systems, motion prediction systems,object type recognition systems, point cloud processing systems, analogsignal processing systems, signal multiplexing systems, data fusionsystems, sensor fusion systems, data filtering systems, statisticalsignal processing systems, signal filtering systems, signal processingsystems, protocol selection systems, storage configuration systems,power management systems, clustering systems, variation systems, machinelearning systems, event prediction systems, autonomous control systems,robotic control systems, robotic process automation systems, datavisualization systems, data normalization systems, data cleansingsystems, data deduplication systems, graph-based data storage systems,intelligent agent systems, object-oriented data storage systems,self-configuration systems, self-healing systems, self-organizingsystems, self-organizing map systems, cost-based routing systems,handshake negotiation systems, entity discovery systems, cybersecuritysystems, biometric systems, natural language processing systems, speechprocessing systems, voice recognition systems, sound processing systems,ultrasound processing systems, artificial intelligence systems, rulesengine systems, workflow automation systems, opportunity discoverysystems, physical modeling systems, testing systems, diagnostic systems,software image propagation systems, peer-to-peer network configurationsystems, RF spectrum management systems, network resource managementsystems, storage management systems, data management systems, intrusiondetection systems, firewall systems, virtualization systems, digitaltwin systems, Internet of Things monitoring systems, routing systems,switching systems, indoor location systems, geolocation systems, parsingsystems, semantic filtering systems, machine vision systems, fuzzy logicsystems, recommendation systems, dialog management systems, and others.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a unified set of adaptiveintelligence systems that provide coordinated intelligence for a set ofdemand management applications, a set of supply chain applications, aset of intelligent product applications and a set of enterprise resourcemanagement applications for a category of goods.

In embodiments, the unified set of adaptive intelligent systems includesa set of artificial intelligence systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of neural networks.In embodiments, the unified set of adaptive intelligent systems includesa set of deep learning systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of model-based systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of expert systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of machine learning systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of rule-based systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of opportunity miners.

In embodiments, the unified set of adaptive intelligent systems includesa set of robotic process automation systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of datatransformation systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of data extraction systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of data loading systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of genetic programming systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of image classification systems. In embodiments, the unified setof adaptive intelligent systems includes a set of video compressionsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of analog-to-digital transformation systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of digital-to-analog transformation systems. In embodiments, theunified set of adaptive intelligent systems includes a set of signalanalysis systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of RF filtering systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of motion predictionsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of object type recognition systems. In embodiments, theunified set of adaptive intelligent systems includes a set of pointcloud processing systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of analog signal processing systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of signal multiplexing systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of data fusion systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of sensor fusion systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of data filtering systems.In embodiments, the unified set of adaptive intelligent systems includesa set of statistical signal processing systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of signal filtering systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of signal processingsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of protocol selection systems. In embodiments, theunified set of adaptive intelligent systems includes a set of storageconfiguration systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of power management systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of clustering systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of variation systems. In embodiments,the unified set of adaptive intelligent systems includes a set ofmachine learning systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of event prediction systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of autonomous control systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of robotic control systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of robotic processautomation systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of data visualization systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of data normalization systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of data cleansing systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of data deduplication systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of graph-based data storagesystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of intelligent agent systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of object-orienteddata storage systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of self-configuration systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of self-healing systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of self-organizing systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of self-organizing mapsystems.

In embodiments, the unified set of adaptive intelligent systems includesa set of cost-based routing systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of handshake negotiationsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of entity discovery systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of cybersecuritysystems.

In embodiments, the unified set of adaptive intelligent systems includesa set of biometric systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of natural language processingsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of speech processing systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of voice recognitionsystems.

In embodiments, the unified set of adaptive intelligent systems includesa set of sound processing systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of ultrasound processingsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of artificial intelligence systems. In embodiments, theunified set of adaptive intelligent systems includes a set of rulesengine systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of workflow automation systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of opportunity discoverysystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of physical modeling systems. In embodiments, the unifiedset of adaptive intelligent systems includes a set of testing systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of diagnostic systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of software image propagationsystems. In embodiments, the unified set of adaptive intelligent systemsincludes a set of peer-to-peer network configuration systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of RF spectrum management systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of network resource management systems. In embodiments, theunified set of adaptive intelligent systems includes a set of storagemanagement systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of data management systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of intrusion detection systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of firewall systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of virtualization systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of digital twin systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of Internet of Things monitoringsystems.

In embodiments, the unified set of adaptive intelligent systems includesa set of routing systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of switching systems. In embodiments,the unified set of adaptive intelligent systems includes a set of indoorlocation systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of geolocation systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of parsing systems. In embodiments, the unified set of adaptiveintelligent systems includes a set of semantic filtering systems. Inembodiments, the unified set of adaptive intelligent systems includes aset of machine vision systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of fuzzy logic systems.

In embodiments, the unified set of adaptive intelligent systems includesa set of recommendation systems. In embodiments, the unified set ofadaptive intelligent systems includes a set of dialog managementsystems. In embodiments, the set of interfaces includes a demandmanagement interface and a supply chain management interface. Inembodiments, the interface is a user interface for a command centerdashboard by which an enterprise orchestrates a set of value chainentities related to a type of product.

In embodiments, the interface is a user interface of a local managementsystem located in an environment that hosts a set of value chainentities. In embodiments, the local management system user interfacefacilitates configuration of a set of network connections for theadaptive intelligence systems. In embodiments, the local managementsystem user interface facilitates configuration of a set of data storageresources for the adaptive intelligence systems. In embodiments, thelocal management system user interface facilitates configuration of aset of data integration capabilities for the adaptive intelligencesystems.

In embodiments, the local management system user interface facilitatesconfiguration of a set of machine learning input resources for theadaptive intelligence systems. In embodiments, the local managementsystem user interface facilitates configuration of a set of powerresources that support the adaptive intelligence systems. Inembodiments, the local management system user interface facilitatesconfiguration of a set of workflows that are managed by the adaptiveintelligence systems.

In embodiments, the interface is a user interface of a mobile computingdevice that has a network connection to the adaptive intelligencesystems.

In embodiments, the interface is an application programming interface.In embodiments, the application programming interface facilitatesexchange of data between the adaptive intelligence systems and acloud-based artificial intelligence system. In embodiments, theapplication programming interface facilitates exchange of data betweenthe adaptive intelligence systems and a real-time operating system of acloud data management platform.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive intelligence systems and acomputational facility of a cloud data management platform.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive intelligence systems and a set ofenvironmental sensors that collect data about an environment that hostsa set of value chain network entities. In embodiments, the applicationprogramming interface facilitates exchange of data between the adaptiveintelligence systems and a set of sensors that collect data about aproduct.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive intelligence systems and a set ofsensors that collect data published by an intelligent product.

In embodiments, the application programming interface facilitatesexchange of data between the adaptive intelligence systems and a set ofsensors that collect data published by a set of Internet of Thingssystems that are disposed in an environment that hosts a set of valuechain network entities.

In embodiments, the set of demand management applications, supply chainapplications, intelligent product applications and enterprise resourcemanagement applications may include, any of the applications mentionedthroughout this disclosure or the documents incorporated herein byreference.

In embodiments, the adaptive intelligent systems layer 614 is configuredto train and deploy artificial intelligence systems to performvalue-chain related tasks. For example, the adaptive intelligent systemslayer 614 may be leveraged to manage a container fleet, design alogistics system, control one or more aspects of a logistics system,select packaging attributes of packages in the value chain, design aprocess to meet regulatory requirements, automate processes to mitigatewaste production (e.g., solid waste or waste water), and/or othersuitable tasks related to the value-chain.

In some of these embodiments, one or more digital twins may be leveragedby the adaptive intelligent systems layer 614. A digital twin may referto a digital representation of a physical object (e.g., an asset, adevice, a product, a package, a container, a vehicle, a ship, or thelike), an environment (e.g., a facility), an individual (e.g., acustomer or worker), or other entity (including any of the value chainnetwork entities 652 described herein), or combination thereof. Furtherexamples of physical assets include containers (e.g., boxes, shippingcontainers, boxes, palates, barrels, and the like), goods/products(e.g., widgets, food, household products, toys, clothing, water, gas,oil, equipment, and the like), components (e.g., chips, boards, screens,chipsets, wires, cables, cards, memory, software components, firmware,parts, connectors, housings, and the like), furniture (e.g., tables,counters, workstations, shelving, etc.), and the like. Examples ofdevices include computers, sensors, vehicles (e.g., cars, trucks,tankers, trains, forklifts, cranes, and the like), equipment, conveyerbelts, and the like. Examples of environments may include facilities(e.g., factories, refineries, warehouses, retail locations, storagebuildings, parking lots, airports, commercial buildings, residentialbuildings, and the like), roads, water ways, cities, countries, landmasses, and the like. Examples of different types of physical assets,devices, and environments are referenced throughout the disclosure.

In embodiments, a digital twin may be comprised of (e.g., via reference,or by partial or complete integration) other digital twins. For example,a digital twin of a package may include a digital twin of a containerand one or more digital twins of one or more respective goods enclosedwithin the container. Taking this example one step further, one or moredigital twins of the packages may be contained in a digital twin of avehicle traversing a digital twin of a road or may be positioned on adigital twin of a shelf within a digital twin of a warehouse, whichwould include digital twins of other physical assets and devices.

In embodiments, the digital representation for a digital twin mayinclude a set of data structures (e.g., classes of objects) thatcollectively define a set of properties, attributes, and/or parametersof a represented physical asset, device, or environment, possiblebehaviors or activities thereof and/or possible states or conditionsthereof, among other things. For example, a set of properties of aphysical asset may include a type of the physical asset, the shapeand/or dimensions of the asset, the mass of the asset, the density ofthe asset, the material(s) of the asset, the physical properties of thematerial(s), the chemical properties of the asset, the expected lifetimeof the asset, the surface of the physical asset, a price of the physicalasset, the status of the physical asset, a location of the physicalasset, and/or other properties, as well as identifiers of other digitaltwins contained within or linked to the digital twin and/or otherrelevant data sources that may be used to populate the digital twin(such as data sources within the management platform described herein orexternal data sources, such as environmental data sources that mayimpact properties represented in the digital twin (e.g., where ambientair pressure or temperature affects the physical dimensions of an assetthat inflates or deflates). Examples of a behavior of a physical assetmay include a state of matter of the physical asset (e.g., a solid,liquid, plasma or gas), a melting point of the physical asset, a densityof the physical asset when in a liquid state, a viscosity of thephysical asset when in a liquid state, a freezing point of the physicalasset, a density of the physical asset when in a solid state, a hardnessof the physical asset when in a solid state, the malleability of thephysical asset, the buoyancy of the physical asset, the conductivity ofthe physical asset, electromagnetic properties of the physical asset,radiation properties, optical properties (e.g., reflectivity,transparency, opacity, albedo, and the like), wave interactionproperties (e.g., transparency or opacity to radio waves, reflectionproperties, shielding properties, or the like), a burning point of thephysical asset, the manner by which humidity affects the physical asset,the manner by which water or other liquids affect the physical asset,and the like. In another example, the set of properties of a device mayinclude a type of the device, the dimensions of the device, the mass ofthe device, the density of the density of the device, the material(s) ofthe device, the physical properties of the material(s), the surface ofthe device, the output of the device, the status of the device, alocation of the device, a trajectory of the device, identifiers of otherdigital twins that the device is connected to and/or contains, and thelike. Examples of the behaviors of a device may include a maximumacceleration of a device, a maximum speed of a device, possible motionsof a device, possible configurations of the device, operating modes ofthe device, a heating profile of a device, a cooling profile of adevice, processes that are performed by the device, operations that areperformed by the device, and the like. Example properties of anenvironment may include the dimensions of the environment, environmentalair pressure, the temperature of the environment, the humidity of theenvironment, the airflow of the environment, the physical objects in theenvironment, currents of the environment (if a body of water), and thelike. Examples of behaviors of an environment may include scientificlaws that govern the environment, processes that are performed in theenvironment, rules or regulations that must be adhered to in theenvironment, and the like.

In embodiments, the properties of a digital twin may be adjusted. Forexample, the temperature of a digital twin, a humidity of a digitaltwin, the shape of a digital twin, the material of a digital twin, thedimensions of a digital twin, or any other suitable parameters may beadjusted to conform to current status data and/or to a predicted statusof a corresponding entity.

In embodiments, a digital twin may be rendered by a computing device,such that a human user can view a digital representation of a set ofphysical assets, devices, or other entities, and/or an environmentthereof. For example, the digital twin may be rendered and provided asan output, or may provide an output, to a display device. In someembodiments, the digital twin may be rendered and output in an augmentedreality and/or virtual reality display. For example, a user may view a3D rendering of an environment (e.g., using monitor or a virtual realityheadset). While doing so, the user may inspect digital twins of physicalassets or devices in the environment. In embodiments, a user may viewprocesses being performed with respect to one or more digital twins(e.g., inventorying, loading, packing, shipping, and the like). Inembodiments, a user may provide input that controls one or moreproperties of a digital twin via a graphical user interface.

In some embodiments, the adaptive intelligent systems layer 614 isconfigured to execute simulations using the digital twin. For example,the adaptive intelligent systems layer 614 may iteratively adjust one ormore parameters of a digital twin and/or one or more embedded digitaltwins. In embodiments, the adaptive intelligent systems layer 614 may,for each set of parameters, execute a simulation based on the set ofparameters and may collect the simulation outcome data resulting fromthe simulation. Put another way, the adaptive intelligent systems layer614 may collect the properties of the digital twin and the digital twinswithin or containing the digital twin used during the simulation as wellas any outcomes stemming from the simulation. For example, in running asimulation on a digital twin of a shipping container, the adaptiveintelligent systems layer 614 can vary the materials of the shippingcontainer and can execute simulations that outcomes resulting fromdifferent combinations. In this example, an outcome can be whether thegoods contained in the shipping container arrive to a destinationundamaged. During the simulation, the adaptive intelligent systems layer614 may vary the external temperatures of the container (e.g., atemperature property of the digital twin of an environment of thecontainer may be adjusted between simulations or during a simulation),the dimensions of the container, the products inside (represented bydigital twins of the products) the container, the motion of thecontainer, the humidity inside the container, and/or any otherproperties of the container, the environment, and/or the contents in thecontainer. For each simulation instance, the adaptive intelligentsystems layer 614 may record the parameters used to perform thesimulation instance and the outcome of the simulation instance. Inembodiments, each digital twin may include, reference, or be linked to aset of physical limitations that define the boundary conditions for asimulation. For example, the physical limitations of a digital twin ofan outdoor environment may include a gravity constant (e.g., 9.8 m/s²),a maximum temperature (e.g., 60 degrees Celsius), a minimum temperature(e.g., −80 degrees Celsius), a maximum humidity (e.g., 110% humidity),friction coefficients of surfaces, maximum velocities of objects,maximum salinity of water, maximum acidity of water, minimum acidity ofwater. Additionally or alternatively, the simulations may adhere toscientific formulas, such as ones reflecting principles or laws ofphysics, chemistry, materials science, biology, geometry, or the like.For example, a simulation of the physical behavior of an object mayadhere to the laws of thermodynamics, laws of motion, laws of fluiddynamics, laws of buoyancy, laws of heat transfer, laws of cooling, andthe like. Thus, when the adaptive intelligent systems layer 614 performsa simulation, the simulation may conform to the physical limitations andscientific laws, such that the outcomes of the simulations mimic realworld outcomes. The outcome from a simulation can be presented to ahuman user, compared against real world data (e.g., measured propertiesof a container, the environment of the container, the contents of thecontainer, and resultant outcomes) to ensure convergence of the digitaltwin with the real world, and/or used to train machine learning models.

FIG. 38 illustrates example embodiments of a system for controllingand/or making decisions, predictions, and/or classification on behalf ofa value chain system 2030. In embodiments, an artificial intelligencesystem 2010 leverages one or more machine-learned models 2004 to performvalue chain-related tasks on behalf of the value chain system 2030and/or to make decisions, classifications, and/or predictions on behalfof the value chain system 2030. In some embodiments, a machine learningsystem 2002 trains the machine learned models 2004 based on trainingdata 2062, outcome data 2060, and/or simulation data 2022. As usedherein, the term machine-learned model may refer to any suitable type ofmodel that is learned in a supervised, unsupervised, or hybrid manner.Examples of machine-learned models include neural networks (e.g., deepneural networks, convolution neural networks, and many others),regression based models, decision trees, hidden forests, Hidden Markovmodels, Bayesian models, and the like. In embodiments, the artificialintelligence system 2010 and/or the value chain system 2030 may provideoutcome data 2060 to the machine-learning system 2002 that relates to adetermination (e.g., decision, classification, prediction) made by theartificial intelligence system 2010 based in part on the one or moremachine-learned models and the input to those models. The machinelearning system may in-turn reinforce/retrain the machine-learned models2004 based on the feedback. Furthermore, in embodiments, themachine-learning system 2002 may train the machine-learning models basedon simulation data 2022 generated by the digital twin simulation system2020. In these embodiments, the digital twin simulation system 2020 maybe instructed to run specific simulations using one or more digitaltwins that represent objects and/or environments that are managed,maintained, and/or monitored by the value chain system. In this way, thedigital twin simulation system 2020 may provide richer data sets thatthe machine-learning system 2002 may use to train/reinforce themachine-learned models. Additionally or alternatively, the digital twinsimulation system 2020 may be leveraged by the artificial intelligencesystem 2010 to test a decision made by the artificial intelligencesystem 2010 before providing the decision to the value chain entity.

In the illustrated example, a machine learning system 2002 may receivetraining data 2062, outcome data 2060, and/or simulation data 2022. Inembodiments, the training data may be data that is used to initiallytrain a model. The training data may be provided by a domain expert,collected from various data sources, and/or obtained from historicalrecords and/or scientific experimentation. The training data 2062 mayinclude quantified properties of an item or environment and outcomesrelating from the quantified properties. In some embodiments, thetraining data may be structured in n-tuples, whereby each tuple includesan outcome and a respective set of properties relating to the outcome.In embodiments, the outcome data 2060 includes real world data (e.g.,data measured or captured from one or more of IoT sensors, value chainentities, and/or other sources). The outcome data may include an outcomeand properties relating to the outcome. Outcome data may be provided bythe value chain system 2030 leveraging the artificial intelligencesystem 2010 and/or other data sources during operation of the valuechain entity system 2010. Each time an outcome is realized (whethernegative or positive), the value chain entity system 2010, theartificial intelligence system 2010, as well as any other data source2050, may output data relating to the outcome to the machine learningsystem 2002. In embodiments, this data may be provided to themachine-learning system via an API of the adaptive intelligent systemslayer 614. Furthermore, in embodiments, the adaptive intelligent systemslayer 614 may obtain data from other types of external data sources thatare not necessarily a value chain entity but may provide insightfuldata. For example, weather data, stock market data, news events, and thelike may be collected, crawled, subscribed to, or the like to supplementthe outcome data (and/or training data and/or simulation data).

In some embodiments, the machine learning system 2002 may receivesimulation data 2022 from the digital twin simulation system 2020.Simulation data 2022 may be any data relating to a simulation using adigital twin. Simulation data 2022 may be similar to outcome data 2060,but the results are simulated results from an executed simulation ratherthan real-world data. In embodiments, simulation data 2022 may includethe properties of the digital twin and any other digital twins that wereused to perform the simulation and the outcomes stemming therefrom. Inembodiments, the digital twin simulation system 2020 may iterativelyadjust the properties of a digital twin, as well as other digital twinsthat are contained or contain the digital twin. During each iteration,the digital twin simulation system 2020 may provide the properties ofthe simulation (e.g., the properties of all the digital twins involvedin the simulation) to the artificial intelligence system 2010, whichthen outputs predictions, classifications, or any other decisions to thedigital twin simulation system 2020. The digital twin simulation system2020 may use the decisions from the artificial intelligence system 2010to execute the simulation (which may result in a series of decisionsstemming from a state change in the simulation). At each iteration, thedigital twin simulation system 2020 may output the properties used torun the simulation to the machine learning system 2002, any decisionsfrom the artificial intelligence system 2010 used by the digital twinsimulation system 2020, and outcomes from the simulation to the machinelearning system 2002, such that the properties, decisions, and outcomesof the simulation are used to further train the model(s) used by theartificial intelligence system during the simulation.

In some embodiments, training data, outcome data 2060, and/or simulationdata 2022 may be fed into a data lake (e.g., a Hadoop data lake). Themachine learning system 2002 may structure the data from the data lake.In embodiments, the machine learning system 2002 may train/reinforce themodels using the collected data to improve the accuracy of the models(e.g., minimize the error value of the model). The machine learningsystem may execute machine-learning algorithms on the collected data(e.g., training data, outcome data, and/or simulation data) to obtainthe model. Depending on the type of model, the machine-learningalgorithm will vary. Examples of learning algorithms/models include(e.g., deep neural networks, convolution neural networks, and manyothers as described throughout this disclosure), statistical models(e.g., regression-based models and many others), decision trees andother decision models, random/hidden forests, Hidden Markov models,Bayesian models, and the like. In collecting data from the digital twinsimulation system 2020, the machine-learning system 2002 may train themodel on scenarios not yet encountered by the value chain system 2030.In this way, the resultant models will have less “unexplored” featurespaces, which may lead to improved decisions by the artificialintelligence system 2010. Furthermore, as digital twins are based partlyon assumptions, the properties of a digital twin may beupdated/corrected when a real-world behavior differs from that of thedigital twin. Examples are provided below.

FIG. 39 illustrates an example of a container fleet management system2070 that interfaces with the adaptive intelligent systems layer 614. Inexample embodiments, a container fleet management system 2070 may beconfigured to automate one or more aspects of the value chain as itapplies to containers and shipping. In embodiments, the container fleetmanagement system 2070 may be include one or more software modules thatare executed by one or more server devices. These software modules maybe configured to select containers to use (e.g., a size of container,the type of the container, the provider of the container, etc.) for aset of one or more shipments, schedule delivery/pickup of container,selection of shipping routes, determining the type of storage for acontainer (e.g., outdoor or indoor), select a location of each containerwhile awaiting shipping, manage bills of lading and/or other suitablecontainer fleet management tasks. In embodiments, the machine-learningsystem 2002 trains one or more models that are leveraged by theartificial intelligence system 2010 to make classifications,predictions, and/or other decisions relating to container fleetmanagement. In example embodiments, a model 2004 is trained to selecttypes of containers given one or more task-related features to maximizethe likelihood of a desired outcome (e.g., that the contents of thecontainer arrive in a timely manner with minimal loss at the lowestpossible cost). As such, the machine-learning system 2002 may train themodels using n-tuples that include the task-related features pertainingto a particular event and one or more outcomes associated with theparticular event. In this example, task-related features for aparticular event (e.g., a shipment) may include, but are not limited to,the type of container used, the contents of the container, properties ofthe container contents (e.g., cost, perishability, temperaturerestrictions, and the like), the source and destination of thecontainer, whether the container is being shipped via truck, rail, orship, the time of year, the cost of each container, and/or otherrelevant features. In this example, outcomes relating to the particularevent may include whether the contents arrived safely, replacement costs(if any) associated with any damage or loss, total shipping time, and/ortotal cost of shipment (e.g., how much it cost to ship container).Furthermore, as international and/or interstate logistics may includemany different sources, destinations, contents, weather conditions, andthe like, simulations that simulate different shipping events may be runto richen the data used to train the model. For instance, simulationsmay be run for different combinations of ports and/or train depots fordifferent combinations of sources, destinations, products, and times ofyear. In this example, different digital twins may be generated torepresent the different combinations (e.g., digital twins of products,containers, and shipping-related environments), whereby one or moreproperties of the digital twins are varied for different simulations andthe outcomes of each simulation may be recorded in a tuple with theproprieties. In this way, the model may be trained on certaincombinations of routes, contents, time of year, container type, and/orcost that may not have been previously encountered in the real-worldoutcome data. Other examples of training a container fleet managementmodel may include a model that is trained to determine where a containershould be stored in a storage facility (e.g., where in a stack, indoorsor outdoors, and/or the like) given the contents of the container, whenthe container needs to be moved, the type of container, the location,the time of year, and the like.

In operation, the artificial intelligence system 2010 may use theabove-discussed models 2004 to make container fleet management decisionson behalf of a container fleet management system 2070 given one or morefeatures relating to a task or event. For example, the artificialintelligence system 2010 may select a type of container (e.g., materialsof the container, the dimensions of the container, the brand of thecontainer, and the like) to use for a particular shipment. In thisexample, the container fleet management system 2070 may provide thefeatures of an upcoming shipment to the artificial intelligence system2010. These features may include what is being shipped (e.g., thetype(s) of goods in the shipment), the size of the shipment, the sourceand destination, the date when the shipment is to be sent off, and/orthe desired date or range of dates for delivery. In embodiments, theartificial intelligence system 2010 may feed these features into one ormore of the models discussed above to obtain one or more decisions.These decisions may include which type of container to use and/or whichshipping routes to use, whereby the decisions may be selected tominimize overall shipping costs (e.g., costs for container andtransit+any replacement costs). The container fleet management system2070 may then initiate the shipping event using the decision(s) made bythe artificial intelligence system 2010. Furthermore, after the shippingevent, the outcomes of the event (e.g., total shipping time, anyreported damages or loss, replacement costs, total costs) may bereported to the machine-learning system 2002 to reinforce the modelsused to make the decisions. Furthermore, in some embodiments, the outputof the container fleet management system 2070 and/or the other valuechain entity data sources 2050 may be used to update one or moreproperties of one or more digital twins via the digital twin system2020.

FIG. 40 illustrates an example of a logistics design system thatinterfaces with the adaptive intelligent systems layer 614. Inembodiments, a logistics design system may be configured to design oneor more aspects of a logistics solution. For example, the logisticsdesign system may be configured to receive one or more logistics factors(e.g., from a user via a GUI). In embodiments, logistics factors mayinclude one or more present conditions, historical conditions, or futureconditions of an organization (or potential organization) that arerelevant to forming a logistics solution. Examples of logistics factorsmay include, but are not limited to the type(s) of products beingproduced/farmed/shipped, features of those products (e.g., dimensions,weights, shipping requirements, shelf life, etc.), locations ofmanufacturing sites, locations of distribution facilities, locations ofwarehouses, locations of customer bases, market penetration in certainareas, expansion locations, supply chain features (e.g., requiredparts/supplies/resources, suppliers, supplier locations, buyers, buyerlocations), and/or the like) and may determine one or more designrecommendations based on the factors. Examples of design recommendationsmay include supply chain recommendations (e.g., proposed suppliers(e.g., resource or parts suppliers), implementations of a smartinventory systems that order on-demand parts from available suppliers,and the like), storage and transport recommendations (e.g., proposedshipping routes, proposed shipping types (e.g., air, freight, truck,ship), proposed storage development (e.g., locations and/or dimensionsof new warehouses), infrastructure recommendations (e.g., updates tomachinery, adding cooled storage, adding heated storage, or the like),and combinations thereof. In embodiments, the logistics design systemdetermines the recommendations to optimize an outcome. Examples ofoutcomes can include manufacturing times, manufacturing costs, shippingtimes, shipping costs, loss rate, environmental impact, compliance to aset of rules/regulations, and the like. Examples of optimizationsinclude increased production throughput, reduced production costs,reduced shipping costs, decreased shipping times, reduced carbonfootprint, and combinations thereof.

In embodiments, the logistics design system may interface with theartificial intelligence system 2010 to provide the logistics factors andto receive design recommendations that are based thereon. Inembodiments, the artificial intelligence system 2010 may leverage one ormore machine-learned models 2004 (e.g., logistics design recommendationsmodels) to determine a recommendation. As will be discussed, a logisticsdesign recommendation model may be trained to optimize one or moreoutcomes given a set of logistics factors. For example, a logisticsdesign recommendation model trained to design supply chains may identifya set of suppliers that can supply a given manufacturer, the location ofthe manufacturer, the supplies needed, and/or other factors. The set ofsuppliers may then be used to implement an on-demand supply sideinventory. In another example, the logistics design recommendation maytake the same features of another manufacturer and recommend thepurchase and use of one or more 3D printers.

In embodiments, the artificial intelligence system 2010 may leverage thedigital twin system 2020 to generate a digital twin of a logisticssystem that implements the logistics design recommendation (and, in someembodiments, alternative systems that implement other designrecommendations). In these embodiments, the digital twin system 1700 mayreceive the design recommendations and may generate a digital twin of alogistics environment that mirrors the recommendations. In embodiments,the artificial intelligence system 2010 may leverage the digital twin ofthe logistics environment to run simulations on the proposed solution.In embodiments, the digital twin system 1700 may display the digitaltwin of the logistics environment to a user via a display device (e.g.,a monitor or a VR headset). In embodiments, the user may view thesimulations in the digital twin. Furthermore, in embodiments, thedigital twin system 1700 may provide a graphical user interface that theuser may interact with to adjust the design of the logistics environmentto adjust the design. The design provided (at least in part) by a usermay also be represented in a digital twin of a logistics environment,whereby the digital twin system 2020 may perform simulations using thedigital twin.

In some embodiments, the simulations run by the digital twin system 1700may be used to train the recommendation models. Furthermore, when thedesign recommendations are implemented by an organization, the logisticssystem of the organization may be configured to report (e.g., viasensors, computing devices, manual human input) outcome datacorresponding to the design recommendations to the machine learningsystem 2002, which may use the outcome data to reinforce the logisticsdesign recommendation models.

FIG. 41 illustrates an example of a packaging design system thatinterfaces with the adaptive intelligent systems layer 614. Inembodiments, the packaging design system may be configured to design oneor more aspects of packaging for a physical object being conveyed in thevalue chain network. In some embodiments, the packaging design systemmay select one or more packaging attributes (e.g., size, material,padding, etc.) of the packaging to optimize one or more outcomesassociated with the transport of the physical object. For example, thepackaging attributes may be selected to reduce costs, decreaseloss/damage, decrease weight, decrease plastic or othernon-biodegradable waste, or the like. In embodiments, the packagingdesign system leverages the artificial intelligence system 2010 toobtain packaging attribute recommendations. In embodiments, thepackaging design system may provide one or more features of the physicalobject. In embodiments, the features of the physical object may includethe dimensions of the physical object, the mass of the physical object,the source of the physical object, one or more potential destinations ofthe physical object, the manner by which the physical object is shipped,and the like. In embodiments, the packaging design system may furtherprovide one or more optimization goals for the package design (e.g.,reduce cost, reduce damage, reduce environmental impact). In response,the artificial intelligence system 2010 may determine one or morerecommended packaging attributes based on the physical asset featuresand the given objective. In embodiments, the packaging design systemreceives the packaging attributes and generates a package design basedthereon. The package design may include a material to be used, theexternal dimensions of the packaging, the internal dimensions of thepackaging, the shape of the packaging, the padding/stuffing for thepackaging, and the like.

In some embodiments, the packaging design system may provide a packagingdesign to the digital twin system 2020, which generates a digital twinof the packaging and physical asset based on the packaging design. Thedigital twin of the packaging and physical asset may be used to runsimulations that test the packaging (e.g., whether the packaging holdsup in shipping, whether the packaging provides adequateinsulation/padding, and the like). In embodiments, the results of thesimulation may be returned to the packaging design system, which mayoutput the results to a user. In embodiments, the user may accept thepackaging design, may adjust the packaging design, or may reject thedesign. In some embodiments, the digital twin system may run simulationson one or more digital twins to test different conditions that thepackage may be subjected to (e.g., outside in the snow, rocking in aboat, being moved by a forklift, or the like). In some embodiments, thedigital twin system may output the results of a simulation to themachine-learning system 2002, which can train/reinforce the packagingdesign models based on the properties used to run the simulation and theoutcomes stemming therefrom.

In embodiments, the machine-learning system 2002 may receive outcomedata from the packaging design system and/or other value chain entitydata sources (e.g., smart warehouses, user feedback, and the like). Themachine-learning system 2002 may use this outcome data totrain/reinforce the packaging design models. Furthermore, in someembodiments, the outcome data may be used by the digital twin system2020 to update/correct any incorrect assumptions used by the digitaltwin system (e.g., the flexibility of a packaging material, the waterresistance of a packaging material, and the like).

FIG. 42 illustrates examples of a waste mitigation system thatinterfaces with the adaptive intelligent systems layer 614. Inembodiments, the waste mitigation system is configured to analyze aprocess within the value chain (e.g., manufacturing of a product, oilrefining, fertilization, water treatment, or the like) to mitigate waste(e.g., solid waste, wastewater, discarded packaging, wasted energy,wasted time, wasted resources, or other waste). In embodiments, thewaste mitigation system may interface with the artificial intelligencesystem 2010 to automate one or more processes to mitigate waste.

In embodiments, the artificial intelligence system 2010 may providecontrol decisions to the waste mitigation system to mitigate solid wasteproduction. Examples of waste production may include excess plastic orother non-biodegradable waste, hazardous or toxic waste (e.g., nuclearwaste, petroleum coke, or the like), and the like. In some of theseembodiments, the artificial intelligence system 2010 may receive one ormore features of the process (or “process features”). Examples ofprocess features may include, but are not limited to, the steps in theprocess, the materials being used, the properties of the materials beingused, and the like. The artificial intelligence system 2010 may leverageone or more machine-learned models to control the process. Inembodiments, the machine-learned models may be trained to classify awaste condition and/or the cause of the waste condition. In some ofthese embodiments, the artificial intelligence system 2010 may determineor select a waste mitigation solution based on the classified wastecondition. For example, in some embodiments, the artificial intelligencesystem 2010 may apply rules-based logic to determine an adjustment tomake to the process to reduce or resolve the waste condition.Additionally, or alternatively, the artificial intelligence may leveragea model that recommends an adjustment to make to the process to reduceor resolve the waste condition.

In embodiments, the artificial intelligence system 2010 may leverage thedigital twin system 2020 to mitigate the waste produced by a process. Inembodiments, the digital twin system 2020 may execute iterativesimulations of the process in a digital twin of the environment in whichthe process is performed. When the simulation is executed, theartificial intelligence system 2010 may monitor the results of thesimulation to determine a waste condition and/or the cause of the wastecondition. During the simulations, the artificial intelligence system2010 may adjust one or more aspects of the process to determine whetherthe adjustments mitigated the waste condition, worsened the wastecondition, or had no effect. When an adjustment is found to mitigate thewaste condition, the artificial intelligence system 2010 may adjustother aspects of the process to determine if an improvement can berealized. In embodiments, the artificial intelligence system 2010 mayperform a genetic algorithm when iteratively adjusting the aspects ofthe process in the digital twin simulations. In these embodiments, theartificial intelligence system 2010 may identify aspects of the processthat can be adjusted to mitigate the waste production.

Smart Project Management Facilities

Referring to FIG. 43 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614 (includingartificial intelligence 1160), a set of data storage facilities orsystems 624, and a set of monitoring facilities or systems 808. Theplatform 604 may support a set of applications 614 (including processes,workflows, activities, events, use cases and applications) for enablingan enterprise to manage a set of value chain network entities 652, suchas from a point of origin to a point of customer use of a product 1510,which may be an intelligent product.

In embodiments, the adaptive intelligence systems layer 614 may furtherinclude a set of automated project management facilities 21006 thatprovide automated recommendations for a set of value chain projectmanagement tasks based on processing current status information, a setof application outputs and/or a set of outcomes 1040 for a set of demandmanagement applications 1502, a set of supply chain applications 1500, aset of intelligent product applications 1510, a set of asset managementapplications 1530 and a set of enterprise resource managementapplications 1520 for a category of goods.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of project managementfacilities that provide automated recommendations for a set of valuechain project management tasks based on processing current statusinformation and a set of outcomes for a set of demand managementapplications, a set of supply chain applications, a set of intelligentproduct applications and a set of enterprise resource managementapplications for a category of goods.

In embodiments, the set of project management facilities are configuredto manage a wide variety of types of projects, such as procurementprojects, logistics projects, reverse logistics projects, fulfillmentprojects, distribution projects, warehousing projects, inventorymanagement projects, product design projects, product managementprojects, shipping projects, maritime projects, loading or unloadingprojects, packing projects, purchasing projects, marketing projects,sales projects, analytics projects, demand management projects, demandplanning projects, resource planning projects and many others.

In embodiments, the project management facilities are configured tomanage a set of procurement projects. In embodiments, the projectmanagement facilities are configured to manage a set of logisticsprojects. In embodiments, the project management facilities areconfigured to manage a set of reverse logistics projects. Inembodiments, the project management facilities are configured to managea set of fulfillment projects.

In embodiments, the project management facilities are configured tomanage a set of distribution projects. In embodiments, the projectmanagement facilities are configured to manage a set of warehousingprojects. In embodiments, the project management facilities areconfigured to manage a set of inventory management projects. Inembodiments, the project management facilities are configured to managea set of product design projects.

In embodiments, the project management facilities are configured tomanage a set of product management projects. In embodiments, the projectmanagement facilities are configured to manage a set of shippingprojects. In embodiments, the project management facilities areconfigured to manage a set of maritime projects. In embodiments, theproject management facilities are configured to manage a set of loadingor unloading projects.

In embodiments, the project management facilities are configured tomanage a set of packing projects. In embodiments, the project managementfacilities are configured to manage a set of purchasing projects. Inembodiments, the project management facilities are configured to managea set of marketing projects. In embodiments, the project managementfacilities are configured to manage a set of sales projects.

In embodiments, the project management facilities are configured tomanage a set of analytics projects. In embodiments, the projectmanagement facilities are configured to manage a set of demandmanagement projects. In embodiments, the project management facilitiesare configured to manage a set of demand planning projects. Inembodiments, the project management facilities are configured to managea set of resource planning projects.

Smart Task Recommendations

Referring to FIG. 45 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614 (includingartificial intelligence 1160), a set of data storage facilities orsystems 624, and a set of monitoring facilities or systems 808.

The platform 604 may support a set of applications 614 (includingprocesses, workflows, activities, events, use cases and applications)for enabling an enterprise to manage a set of value chain networkentities 652, such as from a point of origin to a point of customer useof a product 1510, which may be an intelligent product.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of project managementfacilities that provide automated recommendations for a set of valuechain project management tasks based on processing current statusinformation and a set of outcomes for a set of demand managementapplications, a set of supply chain applications, a set of intelligentproduct applications and a set of enterprise resource managementapplications for a category of goods.

In embodiments, the adaptive intelligent systems layer 614 may furtherinclude a set of process automation facilities 1710 that provideautomated recommendations for a set of value chain process tasks 1710that provide automated recommendations for a set of value chainprocesses based on processing current status information, a set ofapplication outputs and/or a set of outcomes 1040 for a set of demandmanagement applications 1502, a set of supply chain applications 1500, aset of intelligent product applications 1510, a set of asset managementapplications 1530 and a set of enterprise resource managementapplications 1520 for a category of goods. In some examples, the processautomation facilities 1710 may be used with basic rule-based trainingand recommendations. This may relate to following a set of rules that anexpert has articulated such as when a trigger occurs, undertake a task.In another example, the process automation facilities 1710 may utilizedeep learning to observe interactions such as deep learning on outcomesto learn to recommend decisions or tasks that produce a highest returnon investment (ROI) or other outcome-based yield. The process automationfacilities 1710 may be used to provide collaborative filtering such aslook at a set of experts that are most similar in terms of work done andtasks completed being most similar. For example, the underlying softwaremay be used to find customers similar to another set of customers tosell to, make a different offering to, or change price accordingly. Ingeneral, given a set of underlying pattern data, contextually, about acustomer segment, purchasing patterns may be determined for thatcustomer segment such as knowledge of cost and pricing patterns for thatcustomer. This information may be used to learn to focus a next set ofactivities around pricing, promotion, demand management towards an idealthat may be based on deep learning or rules or collaborative filteringtype work trying to leverage off of similar decisions made by similarlysituated people (e.g., recommending movies to a similar cohort ofpeople).

In embodiments, the set of facilities that provide automatedrecommendations for a set of value chain process tasks providerecommendations involving a wide range of types of activities, such as,without limitation, product configuration activities, product selectionactivities for a customer, supplier selection activities, shipperselection activities, route selection activities, factory selectionactivities, product assortment activities, product managementactivities, logistics activities, reverse logistics activities,artificial intelligence configuration activities, maintenanceactivities, product support activities, product recommendationactivities and many others.

In embodiments, the automated recommendations relate to a set of productconfiguration activities. In embodiments, the automated recommendationsrelate to a set of product selection activities for a customer. Inembodiments, the automated recommendations relate to a set of supplierselection activities. In embodiments, the automated recommendationsrelate to a set of shipper selection activities.

In embodiments, the automated recommendations relate to a set of routeselection activities. In embodiments, the automated recommendationsrelate to a set of factory selection activities. In embodiments, theautomated recommendations relate to a set of product assortmentactivities. In embodiments, the automated recommendations relate to aset of product management activities. In embodiments, the automatedrecommendations relate to a set of logistics activities.

In embodiments, the automated recommendations relate to a set of reverselogistics activities. In embodiments, the automated recommendationsrelate to a set of artificial intelligence configuration activities. Inembodiments, the automated recommendations relate to a set ofmaintenance activities. In embodiments, the automated recommendationsrelate to a set of product support activities. In embodiments, theautomated recommendations relate to a set of product recommendationactivities.

Optimized Routing Among Nodes

Referring to FIG. 44 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614 (includingartificial intelligence 1160), a set of data storage facilities orsystems 624, and a set of monitoring facilities or systems 808. Theplatform 604 may support a set of applications 614 (including processes,workflows, activities, events, use cases and applications) for enablingan enterprise to manage a set of value chain network entities 652, suchas from a point of origin to a point of customer use of a product 1510,which may be an intelligent product.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform for a value chain network with amicro-services architecture, a set of interfaces, network connectivityfacilities, adaptive intelligence facilities, data storage facilities,and monitoring facilities that are coordinated for monitoring andmanagement of a set of value chain network entities; and a set ofapplications for enabling an enterprise to manage a set of value chainnetwork entities from a point of origin to a point of customer use;wherein a set of routing facilities generate a set of routinginstructions for routing information among a set of nodes in the valuechain network based on current status information for the value chainnetwork.

In embodiments, the adaptive intelligent systems layer 614 may furtherinclude a set of routing facilities 1720 that generate a set of routinginstructions for routing information among a set of nodes in the valuechain network, such as based on processing current status information1730, a set of application outputs and/or a set of outcomes 1040, orother information collected by or used in the VCNP 102. Routing mayinclude routing for the benefit of a set of demand managementapplications 1502, a set of supply chain applications 1500, a set ofintelligent product applications 1510, a set of asset managementapplications 1530 and a set of enterprise resource managementapplications 1520 for a category of goods.

In embodiments, the set of routing facilities that generate a set ofrouting instructions for routing information among a set of nodes in thevalue chain network use a wide variety of routing systems orconfigurations, such as involving, without limitation, priority-basedrouting, master controller routing, least cost routing, rule-basedrouting, genetically programmed routing, random linear network codingrouting, traffic-based routing, spectrum-based routing, RFcondition-based routing, energy-based routing, latency-sensitiverouting, protocol compatibility based routing, dynamic spectrum accessrouting, peer-to-peer negotiated routing, queue-based routing, andothers.

In embodiments, the routing includes priority-based routing. Inembodiments, the routing includes master controller routing. Inembodiments, the routing includes least cost routing. In embodiments,the routing includes rule-based routing. In embodiments, the routingincludes genetically programmed routing.

In embodiments, the routing includes random linear network codingrouting. In embodiments, the routing includes traffic-based routing. Inembodiments, the routing includes spectrum-based routing.

In embodiments, the routing includes RF condition-based routing. Inembodiments, the routing includes energy-based routing. In embodiments,the routing includes latency-sensitive routing.

In embodiments, the routing includes protocol compatibility-basedrouting.

In embodiments, the routing includes dynamic spectrum access routing. Inembodiments, the routing includes peer-to-peer negotiated routing. Inembodiments, the routing includes queue-based routing.

In embodiments, the status information for the value chain networkinvolves a wide range of states, events, workflows, activities,occurrences, or the like, such as, without limitation, traffic status,congestion status, bandwidth status, operating status, workflow progressstatus, incident status, damage status, safety status, poweravailability status, worker status, data availability status, predictedsystem status, shipment location status, shipment timing status,delivery status, anticipated delivery status, environmental conditionstatus, system diagnostic status, system fault status, cybersecuritystatus, compliance status, demand status, supply status, price status,volatility status, need status, interest status, aggregate status for agroup or population, individual status, and many others.

In embodiments, the status information involves traffic status. Inembodiments, the status information involves congestion status. Inembodiments, the status information involves bandwidth status. Inembodiments, the status information involves operating status. Inembodiments, the status information involves workflow progress status.

In embodiments, the status information involves incident status. Inembodiments, the status information involves damage status. Inembodiments, the status information involves safety status.

In embodiments, the status information involves power availabilitystatus. In embodiments, the status information involves worker status.In embodiments, the status information involves data availabilitystatus.

In embodiments, the status information involves predicted system status.In embodiments, the status information involves shipment locationstatus. In embodiments, the status information involves shipment timingstatus. In embodiments, the status information involves delivery status.

In embodiments, the status information involves anticipated deliverystatus. In embodiments, the status information involves environmentalcondition status.

In embodiments, the status information involves system diagnosticstatus. In embodiments, the status information involves system faultstatus. In embodiments, the status information involves cybersecuritystatus. In embodiments, the status information involves compliancestatus.

Dashboard for Managing Digital Twins

Referring to FIG. 47 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614 (includingartificial intelligence 1160), a set of data storage facilities orsystems 624, and a set of monitoring facilities or systems 808. Theplatform 604 may support a set of applications 614 (including processes,workflows, activities, events, use cases and applications) for enablingan enterprise to manage a set of value chain network entities 652, suchas from a point of origin to a point of customer use of a product 1510,which may be an intelligent product.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a dashboard for managing a setof digital twins, wherein at least one digital twin represents a set ofsupply chain entities, workflows and assets and at least one otherdigital twin represents a set of demand management entities andworkflows.

In embodiments, the VCNP 604 may further include a dashboard 4200 formanaging a set of digital twins 1700. In embodiments, this may includedifferent twins, such as where one digital twin 1700 represents a set ofsupply chain entities, workflows and assets and another digital twin1700 represents a set of demand management entities and workflows. Insome example embodiments, managing a set of digital twins 1700 may referto configuration (e.g., via the dashboard 4200) as described in thedisclosure. For example, the digital twin 1700 may be configured throughuse of a digital twin configuration system to set up and manage theenterprise digital twins and associated metadata of an enterprise, toconfigure the data structures and data listening threads that power theenterprise digital twins, and to configure features of the enterprisedigital twins, including access features, processing features,automation features, reporting features, and the like, each of which maybe affected by the type of enterprise digital twin (e.g., based on therole(s) that it serves, the entities it depicts, the workflows that itsupports or enables and the like). In example embodiments, the digitaltwin configuration system may receive the types of digital twins thatmay be supported for the enterprise, as well as the different objects,entities, and/or states that are to be depicted in each type of digitaltwin. For each type of digital twin, the digital twin configurationsystem may determine one or more data sources and types of data thatfeed or otherwise support each object, entity, or state that is depictedin the respective type of digital twin and may determine any internal orexternal software requests (e.g., API calls) that obtain the identifieddata types or other suitable data acquisitions mechanisms, such aswebhooks, that may configured to automatically receive data from aninternal or external data source In some embodiments, the digital twinconfiguration system may determine internal and/or external softwarerequests that support the identified data types by analyzing therelationships between the different types of data that correspond to aparticular state/entity/object and the granularity thereof. Additionallyor alternatively, a user may define (e.g., via a GUI) the data sourcesand/or software requests and/or other data acquisition mechanisms thatsupport the respective data types that are depicted in a respectivedigital twin. In these example embodiments, the user may indicate thedata source that may be accessed and the types of data to be obtainedfrom the respective data source.

The dashboard may be used to configure the digital twins 1700 for use incollection, processing, and/or representation of information collectedin the platform 604, such as status information 1730, such as for thebenefit of a set of demand management applications 1502, a set of supplychain applications 1500, a set of intelligent product applications 1510,a set of asset management applications 1530 and a set of enterpriseresource management applications 1520 for a category of goods.

In embodiments, the dashboard for managing a set of digital twins,wherein at least one digital twin represents a set of supply chainentities and workflows and at least one other digital twin represents aset of demand management entities and workflows.

In embodiments, the entities and workflows relate to a set of productsof an enterprise. In embodiments, the entities and workflows relate to aset of suppliers of an enterprise. In embodiments, the entities andworkflows relate to a set of producers of a set of products. Inembodiments, the entities and workflows relate to a set of manufacturersof a set of products.

In embodiments, the entities and workflows relate to a set of retailersof a line of products. In embodiments, the entities and workflows relateto a set of businesses involved in an ecosystem for a category ofproducts. In embodiments, the entities and workflows relate to a set ofowners of a set of assets involved in a value chain for a set ofproducts. In embodiments, the entities and workflows relate to a set ofoperators of a set of assets involved in a value chain for a set ofproducts.

In embodiments, the entities and workflows relate to a set of operatingfacilities. In embodiments, the entities and workflows relate to a setof customers. In embodiments, the entities and workflows relate to a setof consumers. In embodiments, the entities and workflows relate to a setof workers.

In embodiments, the entities and workflows relate to a set of mobiledevices. In embodiments, the entities and workflows relate to a set ofwearable devices. In embodiments, the entities and workflows relate to aset of distributors. In embodiments, the entities and workflows relateto a set of resellers.

In embodiments, the entities and workflows relate to a set of supplychain infrastructure facilities. In embodiments, the entities andworkflows relate to a set of supply chain processes. In embodiments, theentities and workflows relate to a set of logistics processes. Inembodiments, the entities and workflows relate to a set of reverselogistics processes.

In embodiments, the entities and workflows relate to a set of demandprediction processes. In embodiments, the entities and workflows relateto a set of demand management processes. In embodiments, the entitiesand workflows relate to a set of demand aggregation processes. Inembodiments, the entities and workflows relate to a set of machines.

In embodiments, the entities and workflows relate to a set of ships. Inembodiments, the entities and workflows relate to a set of barges. Inembodiments, the entities and workflows relate to a set of warehouses.In embodiments, the entities and workflows relate to a set of maritimeports.

In embodiments, the entities and workflows relate to a set of airports.In embodiments, the entities and workflows relate to a set of airways.In embodiments, the entities and workflows relate to a set of waterways.In embodiments, the entities and workflows relate to a set of roadways.

In embodiments, the entities and workflows relate to a set of railways.In embodiments, the entities and workflows relate to a set of bridges.In embodiments, the entities and workflows relate to a set of tunnels.In embodiments, the entities and workflows relate to a set of onlineretailers.

In embodiments, the entities and workflows relate to a set of ecommercesites. In embodiments, the entities and workflows relate to a set ofdemand factors. In embodiments, the entities and workflows relate to aset of supply factors. In embodiments, the entities and workflows relateto a set of delivery systems.

In embodiments, the entities and workflows relate to a set of floatingassets. In embodiments, the entities and workflows relate to a set ofpoints of origin. In embodiments, the entities and workflows relate to aset of points of destination. In embodiments, the entities and workflowsrelate to a set of points of storage.

In embodiments, the entities and workflows relate to a set of points ofproduct usage. In embodiments, the entities and workflows relate to aset of networks. In embodiments, the entities and workflows relate to aset of information technology systems. In embodiments, the entities andworkflows relate to a set of software platforms.

In embodiments, the entities and workflows relate to a set ofdistribution centers. In embodiments, the entities and workflows relateto a set of fulfillment centers. In embodiments, the entities andworkflows relate to a set of containers. In embodiments, the entitiesand workflows relate to a set of container handling facilities.

In embodiments, the entities and workflows relate to a set of customs.In embodiments, the entities and workflows relate to a set of exportcontrol. In embodiments, the entities and workflows relate to a set ofborder control. In embodiments, the entities and workflows relate to aset of drones.

In embodiments, the entities and workflows relate to a set of robots. Inembodiments, the entities and workflows relate to a set of autonomousvehicles. In embodiments, the entities and workflows relate to a set ofhauling facilities. In embodiments, the entities and workflows relate toa set of drones, robots and autonomous vehicles. In embodiments, theentities and workflows relate to a set of waterways. In embodiments, theentities and workflows relate to a set of port infrastructurefacilities.

In embodiments, the set of digital twins may include, for example andwithout limitation, distribution twins, warehousing twins, portinfrastructure twins, shipping facility twins, operating facility twins,customer twins, worker twins, wearable device twins, portable devicetwins, mobile device twins, process twins, machine twins, asset twins,product twins, point of origin twins, point of destination twins, supplyfactor twins, maritime facility twins, floating asset twins, shipyardtwins, fulfillment twins, delivery system twins, demand factors twins,retailer twins, ecommerce twins, online twins, waterway twins, roadwaytwins, roadway twins, railway twins, air facility twins, aircraft twins,ship twins, vehicle twins, train twins, autonomous vehicle twins,robotic system twins, drone twins, logistics factor twins and manyothers.

Microservices Architecture

Referring to FIG. 48 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 614, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of microservices layersincluding an application layer supporting at least one supply chainapplication and at least one demand management application, wherein theapplications of the application layer use a common set of services amonga set of data processing services, data collection services, and datastorage services.

In embodiments, the VCNP 604 may further include a set of microserviceslayers including an application layer supporting at least twoapplications among a set of demand management applications 1502, a setof supply chain applications 1500, a set of intelligent productapplications 1510, a set of asset management applications 1530 and a setof enterprise resource management applications 1520 for a category ofgoods.

A microservices architecture provides several advantages to the platform604. For example, one advantage may be the ability to leverage creationof improved microservices created by others such that developer may onlyneed to define inputs and outputs such that the platform may use readilyadapted services created by others. Also, use of the microservicesarchitecture may provide ability to modularize microservices intocollections that may be used to achieve tasks. For example, a goal todetermine what is happening in a warehouse may be achieved with avariety of microservices with minimal cost such as vision-based service,series of regular prompts that may ask and receive, reading off of eventlogs or feeds, and the like. Each one of these microservices may be adistinct microservice that may be easily plugged in and used. If aparticular microservice does not work effectively, the microservice maybe replaced easily with another service with minimal impact to othercomponents in the platform. Other microservices that may be used includerecommendation service, collaborative filtering service, deep learningwith semi-supervised learning service, etc. The microservicearchitecture may provide modularity at each stage in building a fullworkflow. In an example embodiment, a microservice may be built formultiple applications that may be consumed including shared data steamand anything else enabled by the microservices architecture.

IoT Data Collection Architecture Recommendation of Other Sensors andCameras

Referring to FIG. 49 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of microservices layersincluding an application layer supporting at least one supply chainapplication and at least one demand management application, wherein themicroservice layers include a data collection layer that collectsinformation from a set of Internet of Things resources that collectinformation with respect to supply chain entities and demand managemententities.

Also provided herein are methods, systems, components and other elementsfor an information technology system that may include: a cloud-basedmanagement platform with a micro-services architecture, a set ofinterfaces, network connectivity facilities, adaptive intelligencefacilities, data storage facilities, and monitoring facilities that arecoordinated for monitoring and management of a set of value chainnetwork entities; a set of applications for enabling an enterprise tomanage a set of value chain network entities from a point of origin to apoint of customer use; and a machine learning/artificial intelligencesystem configured to generate recommendations for placing an additionalsensor/and or camera on and/or in proximity to a value chain entity andwherein data from the additional sensor and/or camera feeds into adigital twin that represents a set of value chain entities.

In embodiments, the VCNP 604 may further include a set of microservices,wherein the microservice layers include a monitoring systems and datacollections systems layer 614 having data collection and managementsystems 640 that collect information from a set of Internet of Thingsresources 1172 that collect information with respect to supply chainentities and demand management entities 652. The microservices maysupport various applications among a set of demand managementapplications 1502, a set of supply chain applications 1500, a set ofintelligent product applications 1510, a set of asset managementapplications 1530 and a set of enterprise resource managementapplications 1520 for a category of goods.

In embodiments, the platform 604 may further include a machinelearning/artificial intelligence system 1160 that includes a sensorrecommendation system 1750 that is configured to generaterecommendations for placing an additional sensor 1462 and/or camera onand/or in proximity to a value chain network entity 652. For example, insome embodiments, the sensor recommendation system 1750 may generaterecommendations by using load, array of signals, emergent situations,frequency response, maintenance, diagnosis, etc. Data from theadditional sensor 1462 and/or camera may feed into a digital twin 1700that represents a set of value chain entities 652. In embodiments, theset of Internet of Things resources that collect information withrespect to supply chain entities and demand management entities collectsinformation from entities of any of the types described throughout thisdisclosure and in the documents incorporated by reference herein.

In embodiments, the set of Internet of Things resources may be of a widevariety of types such as, without limitation, camera systems, lightingsystems, motion sensing systems, weighing systems, inspection systems,machine vision systems, environmental sensor systems, onboard sensorsystems, onboard diagnostic systems, environmental control systems,sensor-enabled network switching and routing systems, RF sensingsystems, magnetic sensing systems, pressure monitoring systems,vibration monitoring systems, temperature monitoring systems, heat flowmonitoring systems, biological measurement systems, chemical measurementsystems, ultrasonic monitoring systems, radiography systems, LIDAR-basedmonitoring systems, access control systems, penetrating wave sensingsystems, SONAR-based monitoring systems, radar-based monitoring systems,computed tomography systems, magnetic resonance imaging systems, networkmonitoring systems, or others.

In embodiments, the set of Internet of Things resources includes a setof camera systems. In embodiments, the set of Internet of Thingsresources includes a set of lighting systems. In embodiments, the set ofInternet of Things resources includes a set of machine vision systems.In embodiments, the set of Internet of Things resources includes a setof motion sensing systems.

In embodiments, the set of Internet of Things resources includes a setof weighing systems. In embodiments, the set of Internet of Thingsresources includes a set of inspection systems. In embodiments, the setof Internet of Things resources includes a set of environmental sensorsystems. In embodiments, the set of Internet of Things resourcesincludes a set of onboard sensor systems.

In embodiments, the set of Internet of Things resources includes a setof onboard diagnostic systems. In embodiments, the set of Internet ofThings resources includes a set of environmental control systems. Inembodiments, the set of Internet of Things resources includes a set ofsensor-enabled network switching and routing systems. In embodiments,the set of Internet of Things resources includes a set of RF sensingsystems. In embodiments, the set of Internet of Things resourcesincludes a set of magnetic sensing systems.

In embodiments, the set of Internet of Things resources includes a setof pressure monitoring systems. In embodiments, the set of Internet ofThings resources includes a set of vibration monitoring systems. Inembodiments, the set of Internet of Things resources includes a set oftemperature monitoring systems. In embodiments, the set of Internet ofThings resources includes a set of heat flow monitoring systems. Inembodiments, the set of Internet of Things resources includes a set ofbiological measurement systems.

In embodiments, the set of Internet of Things resources includes a setof chemical measurement systems. In embodiments, the set of Internet ofThings resources includes a set of ultrasonic monitoring systems. Inembodiments, the set of Internet of Things resources includes a set ofradiography systems. In embodiments, the set of Internet of Thingsresources includes a set of LIDAR-based monitoring systems. Inembodiments, the set of Internet of Things resources includes a set ofaccess control systems.

In embodiments, the set of Internet of Things resources includes a setof penetrating wave sensing systems. In embodiments, the set of Internetof Things resources includes a set of SONAR-based monitoring systems. Inembodiments, the set of Internet of Things resources includes a set ofradar-based monitoring systems. In embodiments, the set of Internet ofThings resources includes a set of computed tomography systems. Inembodiments, the set of Internet of Things resources includes a set ofmagnetic resonance imaging systems. In embodiments, the set of Internetof Things resources includes a set of network monitoring systems.

Social Data Collection Architecture

Referring to FIG. 50 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of microservices layersincluding an application layer supporting at least one supply chainapplication and at least one demand management application, wherein themicroservice layers include a data collection layer that collectsinformation from a set of social network sources that provideinformation with respect to supply chain entities and demand managemententities.

In embodiments, the VCNP 604 may further include a set of microserviceslayers that include a data collection layer (e.g., monitoring systemsand data collection systems layer 614) with a social data collectionfacility 1760 that collects information from a set of social networkresources MPVC 1708 that provide information with respect to supplychain entities and demand management entities. The social network datacollection facilities 1760 may support various applications among a setof demand management applications 1502, a set of supply chainapplications 1500, a set of intelligent product applications 1510, a setof asset management applications 1530 and a set of enterprise resourcemanagement applications 1520 for a category of goods. Social networkdata collection (using social network data collection facilities 1760)may be facilitated by a social data collection configuration interface,such as for configuring queries, identifying social data sources ofrelevance, configuring APIs for data collection, routing data toappropriate applications 630, and the like.

Crowdsourcing Data Collection Architecture

Referring to FIG. 51 , an embodiment of the platform 604 is provided. Aswith other embodiments, the platform 604 may employ a micro-servicesarchitecture with the various data handling layers 608, a set of networkconnectivity facilities 642 (which may include or connect to a set ofinterfaces 702 of various layers of the platform 604), a set of adaptiveintelligence facilities or adaptive intelligent systems 1160, a set ofdata storage facilities or systems 624, and a set of monitoringfacilities or systems 808. The platform 604 may support a set ofapplications 614 (including processes, workflows, activities, events,use cases and applications) for enabling an enterprise to manage a setof value chain network entities 652, such as from a point of origin to apoint of customer use of a product 1510, which may be an intelligentproduct.

Thus, provided herein are methods, systems, components and otherelements for an information technology system that may include: acloud-based management platform with a micro-services architecture, aset of interfaces, network connectivity facilities, adaptiveintelligence facilities, data storage facilities, and monitoringfacilities that are coordinated for monitoring and management of a setof value chain network entities; a set of applications for enabling anenterprise to manage a set of value chain network entities from a pointof origin to a point of customer use; and a set of microservices layersincluding an application layer supporting at least one supply chainapplication and at least one demand management application, wherein themicroservice layers include a data collection layer that collectsinformation from a set of crowdsourcing resources that provideinformation with respect to supply chain entities and demand managemententities.

In embodiments, the VCNP 604 may further include a set of microserviceslayers that include a monitoring systems and data collection systemslayer 614 with a crowdsourcing facility 1770 that collects informationfrom a set of crowdsourcing resources that provide information withrespect to supply chain entities and demand management entities. Thecrowdsourcing facilities 1770 may support various applications among aset of demand management applications 1502, a set of supply chainapplications 1500, a set of intelligent product applications 1510, a setof asset management applications 1530 and a set of enterprise resourcemanagement applications 1520 for a category of goods. Crowdsourcing maybe facilitated by a crowdsourcing interface, such as for configuringqueries, setting rewards for information, configuring workflows,determining eligibility for participation, and other elements ofcrowdsourcing.

Value Chain Digital Twin Processing (DTPT)

Referring now to FIG. 52 a set of value chain network digital twins 1700representing a set of value chain network entities 652 is depicted. Thedigital twins 1700 are configured to simulate properties, states,operations, behaviors and other aspects of the value chain networkentities 652. The digital twins 1700 may have a visual user interface,e.g., in the form of 3D models, or may consist of system specificationsor ontologies describing the architecture, including components andtheir interfaces, of the value chain network entities 652. The digitaltwins 1700 may include configuration or condition of the value chainnetwork entities 652, including data records of the past and currentstate of the value chain network entities 652, such as captured throughsensors, through user input, and/or determined by outputs of behavioralmodels that describe the behavior of the value chain network entities652. The digital twins 1700 may be updated continuously to reflect thecurrent condition of the value chain network entities 652, based onsensor data, test and inspection results, conducted maintenance,modifications, etc. The digital twins 1700 may also be configured tocommunicate with a user via multiple communication channels, such asspeech, text, gestures, and the like. For example, a digital twin 1700may receive queries from a user about the value chain network entities652, generate responses for the queries, and communicate such responsesto the user. Additionally or alternatively, digital twins 1700 maycommunicate with one another to learn from and identify similaroperating patterns and issues in other value chain network entities 652,as well as steps taken to resolve those issues. The digital twins 1700may be used for monitoring, diagnostics, simulation, management, remotecontrol, and prognostics, such as to optimize the individual andcollective performance and utilization of value chain network entities652.

For example, machine twins 21010 may continuously capture the keyoperational metrics of the machines 724 and may be used to monitor andoptimize machine performance in real time. Machine twins 21010 maycombine sensor, performance, and environmental data, including insightsfrom similar machines 724, enabling prediction of life span of variousmachine components and informed maintenance decisions. In embodiments,machine twins 21010 may generate an alert or other warning based on achange in operating characteristics of the machine 724. The alert may bedue to an issue with a component of the machine 724. Additionally,machine twins 21010 may determine similar issues that have previouslyoccurred with the machine or similar machines, provide a description ofwhat caused the issues, what was done to address the issues, and explaindifferences between the present issue and the previous issues and whatactions to take to resolve the issue, etc.

Similarly, warehousing twins 1712 may combine a 3D model of thewarehouse with inventory and operational data including the size,quantity, location, and demand characteristics of different products.The warehousing twins 1712 may also collect sensor data in a connectedwarehouse, as well as data on the movement of inventory and personnelwithin the warehouse. Warehousing twins 1712 may help in optimizingspace utilization and aid in identification and elimination of waste inwarehouse operations. The simulation using warehousing twins 1712 of themovement of products, personnel, and material handling equipment mayenable warehouse managers to test and evaluate the potential impact oflayout changes or the introduction of new equipment and new processes.

In embodiments, multiple digital twins of the value chain networkentities 652 may be integrated, thereby aggregating data across thevalue chain network to drive not only entity-level insights but alsosystem-level insights. For example, consider a simple value chainnetwork with an operating facility 712 comprising different machines 724including conveyors, robots, and inspection devices. The operatingfacility digital twin 1172 may need to integrate the data from digitaltwins 1770 of different machines to get a holistic picture of thecomplete conveyor line in the operating facility 712 (e.g., a warehouse,distribution center, or fulfillment center where packages are movedalong a conveyor and inspected before being sent out for delivery. Whilethe digital twin of conveyor line may provide insights about only itsperformance, the composite digital twin may aggregate data across thedifferent machines in the operating facility 712. Thus, it may providean integrated view of individual machines and their interactions withenvironmental factors in the operating facility leading to insightsabout the overall health of the conveyor line within the operatingfacility 712. As another example, the supply factor twins 1650 anddemand factor twins 1640 may be integrated to create a holistic pictureof demand-supply equilibrium for a product 1510. The integration ofdigital twins also enables the querying of multiple value chain networkentities 652 and create a 360-degree view of the value chain network 668and its various systems and subsystems.

It will be apparent that the ability to integrate digital twins of thevalue chain network entities 652 may be used to generate a value chainnetwork digital twin system from a plurality of digital twin subsystemsrepresenting entities selected from among supply chain entities, demandmanagement entities and value chain network entities. For example, amachine digital twin 1770 is comprised of multiple digital twins ofsub-systems and individual components constituting the machine 724. Themachine's digital twin may integrate all such component twins and theirinputs and outputs to build the model of the machine. Also, for example,a distribution facility twins system 1714 may be comprised ofsubsystems, such as warehousing twins 1712, fulfilment twins 1600 anddelivery system twins 1610.

Similarly, the process digital twin may be seen as comprised of digitaltwins of multiple sub-processes representing entities selected fromamong supply chain entities, demand management entities and value chainnetwork entities. For example, the digital twin of a packaging processis comprised of digital twins of sub-processes for picking, moving,inspecting and packing the product. As another example, the digital twinof warehousing process may be seen as comprised of digital twins ofmultiple sub-processes including receiving, storing, picking andshipping of stored inventories.

It will be apparent that a value chain network digital twin system maybe generated from a plurality of digital twin subsystems or conversely adigital twin subsystem may be generated from a digital twin system,wherein at least one of the digital twin subsystem and the digital twinsystem represents entities selected from among supply chain entities,demand management entities and value chain network entities.

Similarly, a value chain network digital twin process may be generatedfrom a plurality of digital twin sub-processes or conversely digitaltwin sub-process generated from a digital twin process wherein at leastone of the digital twin sub-process and the digital twin processrepresents entities selected from among supply chain entities, demandmanagement entities and value chain network entities.

The analytics obtained from digital twins 1700 of the value chainnetwork entities 652 and their interactions with one another provide asystemic view of the value chain network as well as its systems,sub-systems, processes and sub-processes. This may help in generatingnew insights into ways the various systems and processes may be evolvedto improve their performance and efficiency.

In embodiments, the platform 604 and applications 630 may have a systemfor generating and updating a self-expanding digital twin thatrepresents a set of value chain entities. The self-expanding digitaltwin continuously keeps learning and expanding in scope, with more andmore data it collects and scenarios it encounters. As a result, theself-expanding twin can evolve with time and take on more complex tasksand answer more complex questions posed by a user of the self-expandingdigital twin.

In embodiments, the platform 604 and applications 630 may have a systemfor scheduling the synchronization of a physical value chain entity'schanging condition to a digital twin that represents a set of valuechain entities. In embodiments, the synchronization between the physicalvalue chain entity and its digital twin is on a near real-time basis.

In embodiments, the platform 604 and applications 630 may have anapplication programming interface for extracting, sharing, and/orharmonizing data from information technology systems associated withmultiple value chain network entities that contribute to a singledigital twin representing a set of value chain entities.

In embodiments, value chain network management platform 604 may includevarious subsystems that may be implemented as micro services, such thatother subsystems of the system access the functionality of a subsystemproviding a micro service via application programming interface API. Insome embodiments, the various services that are provided by thesubsystems may be deployed in bundles that are integrated, such as by aset of APIs.

In embodiments, value chain network management platform 604 may includea set of microservices for managing a set of value chain networkentities for an enterprise and having a set of processing capabilitiesfor at least one of creating, modifying, and managing the parameters ofa digital twin that is used in the platform to represent a set of valuechain network entities.

Value Chain Digital Twin Kit (DTIB)

The value chain network management platform may provide a digital twinsub-system in the form of an out-of-the-box kit system withself-configuring capabilities. The kit may provide a data-rich andinteractive overview of a set of value chain network entitiesconstituting the sub-system. For example, a supply chain out-of-the-boxdigital twin kit system may represent a set of supply chain entitiesthat are linked to the identity of an owner or operator of the supplychain entities. The owner or operator of the supply chain entity maythen use the kit to get a holistic picture of its complete portfolio.The owner may investigate for information related to various supplychain entities and ask interactive questions from the digital twin kitsystem.

In embodiments, a demand management out-of-the-box digital twin kitsystem may represent a set of demand management entities that are linkedto the identity of an owner or operator of the demand managemententities.

In embodiments, a value chain network digital twin kit system forproviding out-of-the-box, self-configuring capabilities may represent aset of demand management entities and a set of supply chain entitiesthat are linked to the identity of an owner or operator of the demandmanagement entities and the supply chain entities.

In embodiments, a warehouse digital twin kit system for providingout-of-the-box, self-configuring capabilities may represent a set ofwarehouse entities that are linked to the identity of an owner oroperator of the warehouse.

Referring now to FIG. 53 , an example warehouse digital twin kit system5000 is depicted. The warehouse digital twin kit system 5000 includeswarehousing twins in the virtual space 5002 representing models ofwarehouses 654 in the real space 5004.

The warehouse digital twin kit system 5000 allows an owner or operator5008 of the one or more warehouse entities 654 to get complete portfoliooverview of all these entities—existing or in design or construction.The owner 5008 may navigate a wealth of information including warehousephotographs 5010, 3D images 5012, live video feeds 5014 of real-timeconstruction progress and AR or VR renderings 5018 of the warehousingentities 654. The owner 5008 may investigate about the health of one ormore entities 654 and ask interactive questions and search for detailedinformation about one or more warehouse entities 654. The warehousedigital twin kit system 5000 has access to real time dynamic datacaptured by IoT devices and sensors at warehouse entities 654 and may besupported with natural language capabilities enabling it to interactwith the owner 5008 and answer any questions about the condition of thewarehouse entities 654.

In embodiments, warehouse digital twin kit system 5000 may provide theportfolio overview of warehouse entities 654 to owner 5008 in the formof a 3D information map containing all the warehouse entities 654. Owner5008 may select a specific entity on the map and get information aboutinventory, operational and health data from the warehousing twin 1710.Alternatively, the owner 5008 may ask for information about the overallportfolio of warehouse entities 654 owned. The warehouse digital twinkit system 5000 consolidates information from the multiple warehousingtwins 1710 and provides a holistic view. The consolidated view may helpowner 5008 to optimize operations across warehouse entities 654 byadjusting stock locations and staffing levels to match current orforecasted demand. The owner 5008 may also display the information fromwarehouse digital twin kit system 5000 on a website or marketingmaterial to be accessed by any customers, suppliers, vendors and otherpartners.

In embodiments, a container ship digital twin kit system for providingout-of-the-box, self-configuring capabilities may represent a set ofcontainer ship entities that are linked to the identity of an owner oroperator of the container ship.

In embodiments, a port infrastructure digital twin kit system forproviding out-of-the-box, self-configuring capabilities may represent aset of port infrastructure entities that are linked to the identity ofan owner or operator of the port infrastructure.

Value Chain Compatibility Testing (VCCT)

The platform 604 may deploy digital twins 1700 of value chain networkentities 652 for testing the compatibility between different value chainnetwork entities 652 interacting with one another and forming varioussystems and subsystems of the value chain network.

This brings visibility to the compatibility and performance of varioussystems and subsystems within the value chain network before there areany physical impacts. Any incompatibilities or performance deficienciesof different value chain network entities 652 may be highlighted throughdigital models and simulations rather than having to rely on physicalsystems to perform such tests which is both expensive and impractical.

The digital twin 1700 may make use of artificial intelligence systems1160 (including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporatedby reference) for carrying out the compatibility testing in the valuechain network.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a set of vendor components for acontainer ship using a set of digital twins representing the containership and the vendor components.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a set of vendor components for awarehouse using a set of digital twins representing the warehouse andthe vendor components.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a set of vendor components for a portinfrastructure facility using a set of digital twins representing theport infrastructure facility and the vendor components.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a set of vendor components for ashipyard facility using a set of digital twins representing the shipyardfacility and the vendor components.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a container ship and a set of portinfrastructure facilities using a set of digital twins representing thecontainer ship and the port infrastructure facility.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a barge and a set of waterways for anavigation route using a set of digital twins representing the barge andthe set of waterways.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a container ship and a set of cargofor an identified shipment using a set of digital twins representing thecontainer ship and the cargo.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a barge and a set of cargo for anidentified shipment using a set of digital twins representing the bargeand the cargo.

In embodiments, the platform may provide a system for testingcompatibility or configuration of a set of cargo handling infrastructurefacilities and a set of cargo for an identified shipment using a set ofdigital twins representing the cargo handling infrastructure facilitiesand the cargo.

Value Chain Infrastructure Testing (VCIT)

The platform 604 may deploy digital twins 1700 of value chain networkentities 652 to perform stress tests on a set of value chain networkentities. The digital twins may help simulate behavior of value chainnetwork systems and sub-systems in a wide variety of environments. Thestress tests may help run any “what-if” scenarios to understand theimpact of change in relevant parameters beyond normal operating valuesand evaluate the resilience of the infrastructure of value chainnetwork.

The platform 604 may include a system for learning on a training set ofoutcomes, parameters, and data collected from data sources relating to aset of value chain network activities to train artificial intelligencesystems 1160 (including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporated)for performing such stress tests on the value chain network.

In embodiments, the platform may include a system for learning on atraining set of machine outcomes, parameters, and data collected fromdata sources relating to a set of value chain network activities totrain an artificial intelligence/machine learning system to performstress tests on the machine using a digital twin that represents a setof value chain entities.

As described, the value chain network comprises a plurality ofinterrelated sub-systems and sub-processes that manage and control allaspects associated with the production and delivery of a finishedproduct to an end-user—from the acquisition and distribution of rawmaterials between a supplier and a manufacturer, through the delivery,distribution, and storage of materials for a retailer or wholesaler,and, finally, to the sale of the product to an end-user. The complexinterconnected nature of the value chain network means that an adverseevent within one subsystem or one or more value chain entities reflectthrough the entire value chain network.

FIG. 54 is an example method for performing a stress test on the valuechain network. The stress test may comprise a simulation exercise totest the resilience of the value chain network (including itssubsystems) and determine its ability to deal with an adverse scenario,say a natural calamity, a congested route, a change in law, or a deepeconomic recession. Such adverse or stress scenarios may affect one ormore entities or subsystems within the value chain network depending onthe nature of the scenario. Hence, any stress tests would requiresimulating scenarios and analyzing the impact of different scenariosacross different subsystems and on the overall value chain network.

At 5102, all historical and current data related to the value chainnetwork are received. The data may include information related tovarious operating parameters of the value chain network over aparticular historical time period, say last 12 months. The data may alsoprovide information on the typical values of various operatingparameters under normal conditions. Some examples of operatingparameters include: product demand, procurement lead time, productivity,inventory level at one or more warehouses, inventory turnover rates,warehousing costs, average time to transport product from warehouse toshipping terminals, overall cost of product delivery, service levels,etc. At 5104, one or more simulation models of value chain network arecreated based on the data. The simulation models help in visualizing thevalue chain network as a whole and in predicting how changes inoperating parameters affect the operation and performance of the valuechain network. In embodiments, the simulation model may be a sum ofmultiple models of different subsystems of the value chain network.

At 5106, one or more stress scenarios may be simulated by changing oneor more parameters beyond the normal operating values. The simulating ofstress scenarios overcome the limitation of any analysis based only onhistorical data and helps analyze the network performance across a rangeof hypothetical yet plausible stress conditions. The simulation involvesvarying (shocking) one or more parameters while keeping the otherparameters as fixed to analyze the impact of such variations on valuechain network. In embodiments, a single parameter may be varied whilekeeping remaining parameters as fixed. In other embodiments, multipleparameters may be varied simultaneously. At 5108, the outcomes of stressscenario simulations are determined, and the performance of value chainnetwork and its different subsystems is estimated across variousscenarios. At 5110, the data, parameters and outcomes are fed into amachine learning process in the artificial intelligence system 1160 forfurther analysis.

An advantage of generating data through simulations and then trainingmachine learning algorithms on this data is the control this approachprovides on the features in the data as well as volume and frequency ofdata.

In embodiments, the platform may include a system for learning on atraining set of outcomes, parameters, and data collected from datasources relating to a set of value chain network activities to train anartificial intelligence/machine learning system to perform stress testson a physical object using a digital twin that represents a set of valuechain entities.

In embodiments, the platform may include a system for learning on atraining set of outcomes, parameters, and data collected from datasources relating to a set of value chain network activities to train anartificial intelligence/machine learning system to perform stress testson a telecommunications network using a digital twin that represents aset of value chain entities in a connected network of entities and thetelecommunications network.

For example, the telecommunications network may be stress tested forresiliency by deliberately increasing network traffic by generating andsending data packets to a specific target node within thetelecommunications network. Further, the amount of traffic may be variedto create varying load conditions on the target node by manipulating thenumber, rate or amount of data in the data packets. The response fromthe target node may be determined to evaluate how the node performed inthe stress test. The target node may be selected at different parts ofthe telecommunications network for stress testing so as to testrobustness of any portion of the network in any topology. The simulatedstress tests on the telecommunications network may be utilized toidentify vulnerabilities in any portion of a network so that thevulnerability can be rectified before users experience network outagesin a deployed network.

In embodiments, the platform may include a system for using a digitaltwin that represents a set of value chain entities in a demandmanagement environment to perform a set of stress tests on a set ofworkflows in the demand management environment using the digital twin,wherein the stress tests represent impacts in the digital twin ofvarying a set of demand-relevant parameters to levels that exceed normaloperating levels. For example, the demand of a product in the valuechain network may be affected by factors like changes in consumerconfidence, recessions, excessive inventory levels, substitute productpricing, overall market indices, currency exchange changes, etc. Thedemand factors twin 1640 may simulate such scenarios by varying supplyparameters and evaluate the impact of such stresses on the demandenvironments 672. The stress tests performed using the digital twins mayhelp in testing and evaluating the resiliency of the value chain networkboth in cases of over-demand and under-demand.

In embodiments, the platform may include a system for using a digitaltwin that represents a set of value chain entities in the supply chainto perform a set of stress tests on a set of workflows in the supplychain using the digital twin, wherein the stress tests represent impactsin the digital twin of varying a set of supply chain-relevant parametersto levels that exceed normal operating levels. For example, the supplyof a product in the value chain network may be affected by factors likeweather, natural calamities, traffic congestion, regulatory changesincluding taxes and subsidies and border restrictions, etc. The supplyfactors twin 1650 may simulate such scenarios by varying supplyparameters and evaluate the impact of such stresses on the supplyenvironments 670. The stress tests performed using the digital twins mayhelp in testing and evaluating the resiliency of the value chain networkboth in cases of over-supply and under-supply.

Value Chain Incident Management (VCIM)

The platform 604 may deploy digital twins 1700 of value chain networkentities 652 for automatically managing a set of incidents relating to aset of value chain network entities and activities. The incidents mayinclude any events causing disruption to the value chain network likeaccidents, fires, explosions, labor strikes, increases in tariffs,changes in law, changes in market prices (e.g., of fuel, components,materials, or end products), changes in demand, activities of cartels,closures of borders or routes, and/or natural events and/or disasters(including storms, heat waves, winds, earthquakes, floods, hurricanes,tsunamis, etc.), among many others.

Also, the platform 604 may provide real-time visualization and analysisof mobility flows in the value chain network. This may help inquantifying risks, improving visibility and reacting to the disruptionsin the value chain network. For example, real-time visualization of autility flow for shipping activities using a digital twin may help indetecting the occurrence and location of an emergency involving ashipping system and deploying emergency services to the detectedlocation.

In embodiments, the platform may deploy digital twins 1700 of valuechain network entities 652 for more accurate determination of accidentfault. The platform may learn on a training set of accident outcomes,parameters, and data collected from the monitoring layer 614 and datasources of the data storage layer 624 to train artificial intelligencesystem 1160 using a set of digital twins 1700 of involved value chainnetwork entities 652 to determine accident fault. For example, data fromdigital twins of two colliding vehicles may be compared with each otherin addition to data from the drivers, witnesses and police reports todetermine accident fault.

In embodiments, the platform may include a system for learning on atraining set of vehicular event outcomes, parameters, and data collectedfrom data sources related to a set of value chain network entities 652to train artificial intelligence system 1160 to use a digital twins 1700of a selected set of value chain network entities 652 to detect anincidence of fraud. For example, comparing vehicular event data fromdigital twins of vehicles to any insurance claims, contract claims,maritime claims on such vehicles may help in detecting any mismatch inthe two.

In embodiments, the platform may include a system for learning on atraining set of vehicle outcomes, parameters, and data collected fromdata sources related to a set of value chain network entities 652 totrain artificial intelligence system 1160 to use a digital twin 1700 ofa selected set of value chain network entities 652 to detect unreportedabnormal events with respect to selected set of value chain networkentities 652. Consider an example where the digital twin of a vehicleshows an abnormal event like an accident but this event has not beenreported by the driver of the vehicle. The unreported event may be addedto the record of the vehicle and the driver by a lessor of the vehicle.Also, the lessor of the vehicle may charge the lessee for repairs ordiminished value of the vehicle at lease-end and adjust residual valueforecast for the same. Similarly, an insurer may add the unreportedevent to the record of the vehicle and the driver. The reporting may beas detailed as the exact nature, timing, location, fault, etc. of theaccident or just the fact there was unreported accident. Thisinformation may then be used for calculating the insurance premium.

Finally, in case there are multiple entities involved in the accident,the data may be triangulated with the digital twin of another entity forvalidation.

Value Chain Predictive Maintenance (PMVC)

The platform 604 may deploy digital twins 1700 of value chain networkentities 652 to predict when a set of value chain network entitiesshould receive maintenance.

The digital twin may predict the anticipated wear and failure ofcomponents of a system by reviewing historical and current operationaldata thereby reducing the risk of unplanned downtime and the need forscheduled maintenance. Instead of over-servicing or over-maintainingproducts to avoid costly downtime, repairs or replacement, any productperformance is sues predicted by the digital twin may be addressed in aproactive or just-in-time manner.

The digital twins 1700 may collect events or state data about valuechain entities 652 from the monitoring layer 614 and historical or otherdata from selected data sources of the data storage layer 624.Predictive analytics powered by the artificial intelligence system 1160dissect the data, search for correlations, and formulate predictionsabout maintenance need and remaining useful life of a set of value chainentities 652.

The platform 604 may include a system for learning on a training set ofoutcomes, parameters, and data collected from data sources relating to aset of value chain network activities to train artificial intelligencesystems 1160 (including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporated)for performing condition monitoring, anomaly detection, failureforecasting and predictive maintenance of a set of value chain entities652.

In embodiments, the platform may include a system for learning on atraining set of machine maintenance outcomes, parameters, and datacollected from data sources relating to a set of machine activities totrain an artificial intelligence/machine learning system to performpredictive maintenance on a machine using a digital twin of the machine.

In embodiments, artificial intelligence system 1160 may train models,such as predictive models (e.g., various types of neural networks,classification-based models, regression based models, and othermachine-learned models). In embodiments, training can be supervised,semi-supervised, or unsupervised. In embodiments, training can be doneusing training data, which may be collected or generated for trainingpurposes.

An example artificial intelligence system 1160 trains a machinepredictive maintenance model. A predictive maintenance model may be amodel that receives machine related data and outputs one or morepredictions or answers regarding the remaining life of the machine. Thetraining data can be gathered from multiple sources including machinespecifications, environmental data, sensor data, run information,outcome data and notes maintained by machine operators. The artificialintelligence system 1160 takes in the raw data, pre-processes it andapplies machine learning algorithms to generate the predictivemaintenance model. In embodiments, the artificial intelligence system1160 may store the predictive model in a model datastore within datastorage layer 624.

Some examples of questions that the predictive model may answer are:when will the machine fail, what type of failure it will be, what is theprobability that a failure will occur within the next X hours, what isthe remaining useful life of the machine, is the machine behaving in anuncharacteristic manner, which machine requires maintenance mosturgently and the like.

The artificial intelligence system 1160 may train multiple predictivemodels to answer different questions. For example, a classificationmodel may be trained to predict failure within a given time window,while a regression model may be trained to predict the remaining usefullife of the machine.

In embodiments, training may be done based on feedback received by thesystem, which is also referred to as “reinforcement learning.” Inembodiments, the artificial intelligence system 1160 may receive a setof circumstances that led to a prediction (e.g., attributes of amachine, attributes of a model, and the like) and an outcome related tothe machine and may update the model according to the feedback.

In embodiments, artificial intelligence system 1160 may use a clusteringalgorithm to identify the failure pattern hidden in the failure data totrain a model for detecting uncharacteristic or anomalous behavior. Thefailure data across multiple machines and their historical records maybe clustered to understand how different patterns correlate to certainwear-down behavior and develop a maintenance plan resonant with thefailure.

In embodiments, artificial intelligence system 1160 may output scoresfor each possible prediction, where each prediction corresponds to apossible outcome. For example, in using a predictive model used todetermine a likelihood that a machine will fail in the next one week,the predictive model may output a score for a “will fail” outcome and ascore for a “will not fail” outcome. The artificial intelligence system1160 may then select the outcome with the greater score as theprediction. Alternatively, the system 1160 may output the respectivescores to a requesting system. In embodiments, the output from system1160 includes a probability of the prediction's accuracy.

FIG. 55 is an example method used by machine twin 1770 for detectingfaults and predicting any future failures of machine 724.

At 5202, a plurality of streams of machine related data from multipledata sources are received at the machine twin 1770. This includesmachine specifications like mechanical properties, data from maintenancerecords, operating data collected from the sensors, historical dataincluding failure data from multiple machines running at different timesand under different operating conditions and so on. At 5205, the rawdata is cleaned by removing any missing or noisy data, which may occurdue to any technical problems in the machine at the time of collectionof data. At 5208, one or more models are selected for training bymachine twin 1770. The selection of model is based on the kind of dataavailable at the machine twin 1770 and the desired outcome of the model.For example, there may be cases where failure data from machines is notavailable, or only a limited number of failure datasets exist because ofregular maintenance being performed. Classification or regression modelsmay not work well for such cases and clustering models may be mostsuitable. As another example, if the desired outcome of the model isdetermining current condition of the machine and detecting any faults,then fault detection models may be selected, whereas if the desiredoutcome is predicting future failures then remaining useful lifeprediction model may be selected. At 5210, the one or more models aretrained using training dataset and tested for performance using testingdataset. At 5212, the trained model is used for detecting faults andpredicting future failure of the machine on production data.

FIG. 56 is an example embodiment depicting the deployment of machinetwins 21010 perform predictive maintenance on machines 724. Machine twin1770 receives data from data storage systems 624 on a real-time or nearreal-time basis. The data storage systems 624 may store different typesof data in different datastores. For example, machine datastore 5202 maystore data related to machine identification and attributes, machinestate and event data, data from maintenance records, historicaloperating data, notes from machine operator, etc. Sensor datastore 5204may store sensor data from operation such as temperature, pressure, andvibration that may be stored as signal or time series data. Failuredatastore 5310 may store failure data from machine 724 or similarmachines running at different times and under different operatingconditions. Model datastore 5312 may store data related to differentpredictive models including fault detection and remaining lifeprediction models.

Machine twin 1770 then coordinates with artificial intelligence systemto select one or more of models based on the kind and quality ofavailable data and the desired answers or outcomes. For example,physical models 5320 may be selected if the intended use of machine twin1770 is to simulate what-if scenarios and predict how the machine willbehave under such scenarios. Fault Detection and Diagnostics Models 5322may be selected to determine the current health of the machine and anyfault conditions. A simple fault detection model may use one or morecondition indicators to distinguish between regular and faulty behaviorsand may have a threshold value for the condition indicator that isindicative of a fault condition when exceeded. A more complex model maytrain a classifier to compare the value of one or more conditionindicators to values associated with fault states and returns theprobability of presence of one or more fault states.

Remaining Useful Life (RUL) Prediction models 5324 are used forpredicting future failures and may include degradation models 5326,survival models 5328 and similarity models 5330. An example RULprediction model may fit the time evolution of a condition indicator andpredicts how long it will be before the condition indicator crosses somethreshold value indicative of a failure. Another model may compare thetime evolution of the condition indicator to measured or simulated timeseries from similar systems that ran to failure.

In embodiments, a combination of one or more of these models may beselected by the machine twin 1770.

Artificial Intelligence system 1160 may include machine learningprocesses 5340, clustering processes 5342, analytics processes 5344 andnatural language processes 5348. Machine learning processes 5340 workwith machine twin 1770 to train one or more models as identified above.An example of such machine learned model is the RUL prediction model5324. The model 5324 may be trained using training dataset 5350 from theData Storage Systems 624. The performance of the model 5324 andclassifier may then be tested using testing dataset 5350.

Clustering processes 5342 may be implemented to identify the failurepattern hidden in the failure data to train a model for detectinguncharacteristic or anomalous behavior. The failure data across multiplemachines and their historical records may be clustered to understand howdifferent patterns correlate to certain wear-down behavior. Analyticsprocesses 5344 perform data analytics on various data to identifyinsights and predict outcomes. Natural language processes 4348coordinate with machine twin 1770 to communicate the outcomes andresults to the user of machine twin 1770.

The outcomes 5360 may be in the form of modeling results 5362, alertsand warnings 5364 or remaining useful life (RUL) predictions 5368.Machine twin 1770 may communicate with a user via multiple communicationchannels such as speech, text, gestures to convey outcomes 5360.

In embodiments, models may then be updated or reinforced based on themodel outcomes 5360. For example, the artificial intelligence system mayreceive a set of circumstances that led to a prediction of failure andthe outcome and may update the model based on the feedback.

In embodiments, the platform may include a system for learning on atraining set of ship maintenance outcomes, parameters, and datacollected from data sources relating to a set of ship activities totrain an artificial intelligence/machine learning system to performpredictive maintenance on a ship using a digital twin of the ship.

In embodiments, the platform may include a system for learning on atraining set of barge maintenance outcomes, parameters, and datacollected from data sources relating to a set of barge activities totrain an artificial intelligence/machine learning system to performpredictive maintenance on a barge using a digital twin of the barge.

In embodiments, the platform may include a system for learning on atraining set of port maintenance outcomes, parameters, and datacollected from data sources relating to a set of port activities totrain an artificial intelligence/machine learning system to performpredictive maintenance on a port infrastructure facility using a digitaltwin of the port infrastructure facility.

In embodiments, the platform may include a system for learning on atraining set of repair outcomes, parameters, and data collected fromdata sources related to a set of value chain entities to train anartificial intelligence/machine learning system to use a digital twin ofa selected set of value chain entities to estimate the cost of repair ofa damaged object.

In embodiments, the platform may include a system for learning on atraining set of infrastructure outcomes, parameters, and data collectedfrom data sources to train an artificial intelligence/machine learningsystem to predict deterioration of infrastructure using a digital twinof the infrastructure.

In embodiments, the platform may include a system for learning on atraining set of natural hazard outcomes, parameters, and data collectedfrom data sources relating to a set of shipping activities to train anartificial intelligence/machine learning system to model natural hazardrisks for a set of shipping infrastructure facilities using a digitaltwin of a city.

In embodiments, the platform may include a system for learning on atraining set of maintenance outcomes, parameters, and data collectedfrom data sources relating to a set of shipping activities to train anartificial intelligence/machine learning system to monitor shippinginfrastructure maintenance activities for a set of shippinginfrastructure facilities using a digital twin of the set of facilities

In embodiments, the platform may include a system for learning on atraining set of maintenance outcomes, parameters, and data collectedfrom data sources relating to a set of shipping activities to train anartificial intelligence/machine learning system to detect the occurrenceand location of a maintenance issue using a digital twin of a set ofshipping infrastructure facilities and having a system for automaticallydeploying maintenance services to the detected location.

Referring to FIG. 57 , the platform 604 may include, integrate,integrate with, manage, control, coordinate with, or otherwise handlecustomer digital twins 5502 and/or customer profile digital twins 1730.

Customer digital twins 5502 may represent evolving, continuously updateddigital representations of value chain network customers 662. Inembodiments, value chain network customers 662 include consumers,licensees, businesses, enterprises, value-added resellers and otherresellers, distributors, retailers (including online retailers, mobileretailers, conventional brick and mortar retailers, pop-up shops and thelike), end users, and others who may purchase, license, or otherwise usea category of goods and/or related services.

Customer profile digital twins 1730, on the other hand, may representone or more demographic (age, gender, race, marital status, number ofchildren, occupation, annual income, education level, living status(homeowner, renter, and the like) psychographic, behavioral, economic,geographic, physical (e.g., size, weight, health status, physiologicalstate or condition, or the like) or other attributes of a set ofcustomers. In embodiments, customer profile digital twins 1730 may beenterprise customer profile digital twins that represent attributes of aset of enterprise customers. In embodiments, a customer profilingapplication may be used to manage customer profiles 5504 based onhistorical purchasing data, loyalty program data, behavioral trackingdata (including data captured in interactions by a customer with anintelligent product 1510), online clickstream data, interactions withintelligent agents, and other data sources.

Customers 662 can be depicted in a set of one or more customer digitaltwins 5502, such as by populating the customer digital twin 1730 withvalue chain network data objects 1004, such as event data 1034, statedata 1140, or other data with respect to value chain network customers662. Likewise, customer profiles 5504 can be depicted in a set of one ormore customer profile digital twins 1730, such as by populating thecustomer profile digital twins 1730 with value chain network dataobjects 1004, such as described throughout this disclosure.

Customer digital twins 5502 and customer profile digital twins 1730 mayallow for modeling, simulation, prediction, decision-making,classification, and the like.

Where customers 662 are consumers, for example, the respective customerdigital twins 1730 may be populated with identity data, account data,payment data, contact data, age data, gender data, race data, locationdata, demographic data, living status data, mood data, stress data,behavior data, personality data, interest data, preference data, styledata, medical data, physiological data, phycological data, physicalattribute data, education data, employment data, salary data, net worthdata, family data, household data, relationship data, pet data,contact/connection data (such as mobile phone contacts, social mediaconnections, and the like), transaction history data, political data,travel data, product interaction data, product feedback data, customerservice interaction data (such as a communication with a chatbot, or atelephone communication with a customer service agent at a call center),fitness data, sleep data, nutrition data, software program interactionobservation data 1500 (e.g., by customers interacting with varioussoftware interfaces of applications 630 involving value chain entities652) and physical process interaction observation data 1510 (e.g., bywatching customers interacting with products or other value chainentities 652), and the like.

In another example, where customers 662 are enterprises or businesses,the customer digital twin 1730 may be populated with identity data,account data, payment data, transaction data, product feedback data,location data, revenue data, enterprise type data, product and/orservice offering data, worker data (such as identity data, role data,and the like), and other enterprise-related attributes.

Customer digital twins and customer profile digital twins 1730 mayinclude a set of components, processes, services, interfaces, and otherelements for development and deployment of digital twin capabilities forvisualization of value chain network customers 662 and customer profiles5504 as well as for coordinated intelligence (including artificialintelligence system 1160, edge intelligence, analytics and othercapabilities) and other value-added services and capabilities that areenabled or facilitated with digital twins.

In embodiments, the customer digital twins 5502 and customer profiledigital twins 1730 may take advantage of the presence of multipleapplications 630 within the value chain management platform 604, suchthat a pair of applications may share data sources (such as in the datastorage layer 624) and other inputs (such as from the monitoring layer614) that are collected with respect to value chain entities 652, aswell as sharing events, state information and outputs, whichcollectively may provide a much richer environment for enriching contentin the digital twins, including through use of artificial intelligencesystem 1160 (including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporatedby reference) and through use of content collected by the monitoringlayer 614 and data collection systems 640.

An environment for development of a customer digital twin 5502 mayinclude a set of interfaces for developers in which a developer mayconfigure an artificial intelligence system 1160 to take inputs fromselected data sources of the data storage layer 624 and events or otherdata from the monitoring systems layer 614 and supply them for inclusionin a customer digital twin 5502. A customer digital twin developmentenvironment may be configured to take outputs and outcomes from variousapplications 630. In embodiments, a customer digital twin 1730 may beprovided for the wide range of value chain network applications 630mentioned throughout this disclosure and the documents incorporatedherein by reference.

In embodiments, the customer digital twin 5502 may be rendered by acomputing device, such that a user can view a digital representation ofthe customer 714. For example, a customer digital twin 5502 may berendered and output to a display device. In another example, a 5502 maybe rendered in a three-dimensional environment and viewed using avirtual reality headset.

An environment for development of the customer profile digital twin 1730may include a set of interfaces for developers in which a developer mayconfigure an artificial intelligence system 1160 to take inputs fromselected data sources of the data storage layer 624 and events or otherdata from the monitoring systems layer 614 and supply them for inclusionin the customer profile digital twin 1730. A customer profile digitaltwin development environment may be configured to take outputs andoutcomes from various applications 630. In embodiments, the customerprofile digital twin 1730 may be provided for the wide range of valuechain network applications 630 mentioned throughout this disclosure andthe documents incorporated herein by reference.

In embodiments, the adaptive intelligent systems layer 614 is configuredto train and implement artificial intelligence systems to perform tasksrelated to the value chain network 668 and/or value chain networkentities 652. For example, the adaptive intelligent systems layer 614may be leveraged to recommend products, enhance customer experience,select advertising attributes for advertisements relating to value chainproducts and/or services, and/or other appropriate value-chain tasks.

In embodiments, the customer profile digital twin 1730 or other customerdigital twin may be created interactively and cooperatively with acustomer, such as by allowing a customer to request, select, modify,delete, or otherwise influence a set of properties, states, behaviors,or other aspects represented in the digital twin 1730. For example, acustomer could refine sizes (e.g., shoe size, dress size, shirt size,pant size, and the like), indicate interests and needs (e.g., what thecustomer is interested in buying), indicate behaviors (e.g., projectsplanned by an enterprise), update current states (e.g., to reflectchanges), and the like. A version of the digital twin 1730 may thus bemade available to a customer, such as in a graphical user interface,where the customer may manipulate one or more aspects of the digitaltwin 1730, request changes, and the like. In embodiments, multipleversions of a digital twin 1730 may be maintained for a given customer,such as a version for customer review, an internal version for anenterprise or host, a version for each of a specific set of brands(e.g., where a customer's appropriate clothing sizes vary by brand), apublic version (such as one shared with a customer's social network forfeedback, such as from friends), a private version (such as one where acustomer is provided complete control over features and properties), asimulation version, a real-time version, and the like. In embodiments,the adaptive intelligent systems layer 614 is configured to leverage thecustomer digital twins 5502, customer profile digital twins 1730, and/orother digital twins 1700 of other value chain network entities 652. Inembodiments, the adaptive intelligent systems layer 614 is configured toperform simulations using the customer digital twins 5502, customerprofile digital twins 1730, and/or digital twins of other value chainnetwork entities 652. For example, the adaptive intelligent systemslayer 614 may vary one or more features of a product digital twin 1780as its use is simulated by a customer digital twin 1730.

In embodiments, a simulation management system 5704 may set up,provision, configure, and otherwise manage interactions and simulationsbetween and among digital twins 1700 representing value chain entities652.

In embodiments, the adaptive intelligent systems layer 614 may, for eachset of features, execute a simulation based on the set of features andmay collect the simulation outcome data resulting from the simulation.For example, in executing a simulation involving the interactions of anintelligent product digital twin 1780 representing an intelligentproduct 1510 and a customer digital twin 1730, the adaptive intelligentsystems layer 614 can vary the dimensions of the intelligent productdigital twin 1780 and can execute simulations that generate outcomes ina simulation management system 5704. In this example, an outcome can bean amount of time taken by a customer digital twin 5502 to complete atask using the intelligent product digital twin 1780. During thesimulations, the adaptive intelligent systems layer 614 may vary theintelligent product digital twin 1780 display screen size, availablecapabilities (processing, speech recognition, voice recognition, touchinterfaces, remote control, self-organization, self-healing, processautomation, computation, artificial intelligence, data storage, and thelike), materials, and/or any other properties of the intelligent productdigital twin 1780. Simulation data 5710 may be created for eachsimulation and may include feature data used to perform the simulations,as well as outcome data. In the example described above, the simulationdata 5710 may be the properties of the customer digital twin 5502 andthe intelligent product digital twin 1780 that were used to perform thesimulation and the outcomes resulting therefrom. In embodiments, amachine learning system 5720 may receive training data 5730, outcomedata 5740, simulation data 5710, and/or data from other types ofexternal data sources 5702 (weather data, stock market data, sportsevent data, news event data, and the like). In embodiments, this datamay be provided to the machine-learning system 5720 via an API of theadaptive intelligent systems layer 614. The machine learning system 5720may train, retrain, or reinforce machine leaning models 5750 using thereceived data (training data, outcome data, simulation data, and thelike).

FIG. 58 illustrates an example of an advertising application thatinterfaces with the adaptive intelligent systems layer 614. In exampleembodiments, the advertising application may be configured automateadvertising-related tasks for a value chain product or service.

In embodiments, the machine-learning system 5720 trains one or moremodels 5750 that are leveraged by the artificial intelligence system1160 to make classifications, predictions, and/or other decisionsrelating to advertisements for a set of value chain products and/orservices.

In example embodiments, a model 5750 is trained to select advertisementfeatures to optimize one or more outcomes (e.g., maximize product salesfor a product 1510 in the value chain network 668). The machine-learningsystem 5720 may train the models 5750 using n-tuples that include thefeatures pertaining to advertisements and one or more outcomesassociated with the advertisements. In this example, features for anadvertisement may include, but are not limited to, product and/orservice category advertised, advertised product features (price, productvendor, and the like), advertised service features, advertisement type(television, radio, podcast, social media, e-mail or the like),advertisement length (10 seconds, 30 seconds, or the like),advertisement timing (in the morning, before a holiday, and the like),advertisement tone (comedic, informational, emotional, or the like),and/or other relevant advertisement features. In this example, outcomesrelating to the advertisement may include product sales, total cost ofthe advertisement, advertisement interaction measures, and the like. Inthis example, one or more digital twins 1700 may be used to simulate thedifferent arrangements (e.g., digital twins of advertisements,customers, customer profiles, and environments), whereby one or moreproperties of the digital twins are varied for different simulations andthe outcomes of each simulation may be recorded in a tuple with theproprieties. Other examples of training advertising models may include amodel that is trained to generate advertisements for value chainproducts 650, a model that is trained to manage an advertising campaignfor value chain products 650, and the like. In operation, the artificialintelligence system 1160 may use such models 5750 to make advertisementdecisions on behalf of an advertising application 5602 given one or morefeatures relating to an advertising-related task or event. For example,the artificial intelligence system 1160 may select a type ofadvertisement (e.g., social media, podcast, and the like) to use for avalue chain product 1510. In this example, the advertising application5602 may provide the features of the product to artificial intelligencesystem 1160. These features may include product vendor, the price of theproduct, and the like. In embodiments, the artificial intelligencesystem 1160 may insert these features into one or more of the models5750 to obtain one or more decisions, which may include which type ofadvertisement to use. In embodiments, the artificial intelligence system1160 may leverage the customer digital twins 5502 and/or customerprofile digital twins 1730 to run simulations on the one or moredecisions and generate simulation data 5710. The machine learning system5720 may receive the simulation data 5710 and other data as describedthroughout this disclosure to retrain or reinforce machine leaningmodels. In embodiments, the customer digital twins 5502, customerprofile digital twins 1730, and other digital twins 1700 may beleveraged by the artificial intelligence system 1160 to simulate adecision made by the artificial intelligence system 1160 beforeproviding the decision to the value chain entity 652. In the presentexample, the customer profile digital twins 1730 may be leveraged by theartificial intelligence system 1160 to simulate decisions made by theartificial intelligence system 1160 before providing the decision to theadvertising application 5602. In embodiments, where simulation outcomesare unacceptable, simulation data 5710 may be reported to the machinelearning system 5720, which may use the received data to re-trainmachine learning models 5750, which may then be leveraged by theartificial intelligence system 1160 to make a new decision. Theadvertising application 824 may initiate an advertising event using thedecision(s) made by the artificial intelligence system 1160. Inembodiments, after the advertising event, the outcomes of the event(e.g., product sales) may be reported to the machine-learning system5720 to reinforce the models 5750 used to make the decisions.Furthermore, in some embodiments, the output of the advertisingapplication and/or the other value chain entity data sources may be usedto update one or more properties of customer digital twins 5502,customer profile digital twins 1730 and/or other digital twins 1700.

FIG. 59 illustrates an example of an e-commerce application 5604integrated with the adaptive intelligent systems layer 614. Inembodiments, an e-commerce application 5604 may be configured togenerate product recommendations for value chain customers 662. Forexample, the ecommerce application 5604 may be configured to receive oneor more product features for a value chain network product 1510.Examples of product features may include, but are not limited to producttypes, product capabilities, product price, product materials, productvendor, and the like. In embodiments, the e-commerce application 5604determines recommendations to optimize an outcome. Examples of outcomescan include software interaction observations (such as mouse movements,mouse clicks, cursor movements, navigation actions, menu selections, andmany others), such as logged and/or tracked by software interactionobservation system 1500, purchase of the product by a customer 714, andthe like. In embodiments, the e-commerce application 5604 may interfacewith the artificial intelligence system 1160 to provide product featuresand to receive product recommendations that are based thereon. Inembodiments, the artificial intelligence system 1160 may utilize one ormore machine-learned models 5750 to determine a recommendation. In someembodiments, the simulations run by the customer digital twin 1730 maybe used to train the product recommendation machine-learning models.

FIG. 60 is a schematic illustrating an example of demand managementapplication 824 integrated with the adaptive intelligent systems layer614. In embodiments, the artificial intelligence system 1160 may usemachine-learning models 5750 trained to make demand management decisionsfor a demand environment 672 on behalf of a demand managementapplication 824 given one or more demand factors 644. Demand factors 644may include product type, product capabilities, product price, productmaterials, time of year, location, and the like. In embodiments, theartificial intelligence system 1160 may determine a demand managementdecision for a value chain product 1510. For example, the artificialintelligence system 1160 may generate a demand management decisionrelating to how many printer ink cartridges should be supplied to aparticular region for an upcoming month. In this example, the demandmanagement system 824 may provide the demand factors 644 to artificialintelligence system 1160. In embodiments, the artificial intelligencesystem 1160 may insert these factors 644 into one or moremachine-learning models 5750 to obtain one or more demand managementdecisions. These decisions may include the volume of ink cartridgesshould be sent to the select region during the select month.

In embodiments, the artificial intelligence system 1160 may leverage thecustomer profile digital twins 1730 to run simulations on the proposeddecisions related to the demand management. The demand managementapplication 824 may then initiate an ink resupply event using thedecision(s) made by the artificial intelligence system 1160.Furthermore, after the ink resupply event, the outcomes of the event(e.g., ink cartridge sales) may be reported to the machine-learningsystem 5720 to reinforce the models used to make the decisions.Furthermore, in some embodiments, the output of the demand managementsystem 824 and/or the other value chain entity data sources may be usedto update one or more properties of customer profile digital twins 1730and/or other digital twins 1700.

In embodiments, an API enables users to access the customer digitaltwins 5502 and/or customer profile digital twins 1730. In embodiments,an API enables users to receive one or more reports related to thedigital twins.

The platform 604 may include, integrate, integrate with, manage,control, coordinate with, or otherwise handle household demand digitaltwins 5902. Household demand digital twins 5902 may be a digitalrepresentation of a household demand for a product category or for a setof product categories.

An environment for development of a household demand digital twin 5902may include a set of interfaces for developers in which a developer mayconfigure an artificial intelligence system 1160 to take inputs fromselected data sources of the data storage layer 624 and events or otherdata from the monitoring systems layer 614 and supply them for inclusionin a household demand digital twin 5902. A household demand digital twindevelopment environment may be configured to take outputs and outcomesfrom various applications 630. In embodiments, a household demanddigital twin 5902 may be provided for the wide range of value chainnetwork applications 630 mentioned throughout this disclosure and thedocuments incorporated herein by reference.

In embodiments, a digital twin 1700 may be generated from other digitaltwins. For example, a customer digital twin 5502 may be used to generatean anonymized customer digital twin 5902. The platform may include,integrate, integrate with, manage, control, coordinate with, orotherwise handle anonymized customer digital twins 5902. Anonymizedcustomer digital twins 5902 may be an anonymized digital representationof a customer 714. In embodiments, anonymized customer digital twins5902 are not populated with personally identifiable information but mayotherwise be populated using the same data sources as its correspondingcustomer digital twin 5502.

In embodiments, an environment for development of an anonymized customerdigital twin 1730 may include a set of interfaces for developers inwhich a developer may configure an artificial intelligence system 1160to take inputs from selected data sources of the data storage layer 624and events or other data from the monitoring systems layer 614 andsupply them for inclusion in an anonymized customer digital twin 5902.An anonymized digital twin development environment may be configured totake outputs and outcomes from various applications 630. In embodiments,an anonymized customer digital twin 5902 may be provided for the widerange of value chain network applications 630 mentioned throughout thisdisclosure and the documents incorporated herein by reference.

In embodiments, the anonymized customer digital twin 5902 comprises anAPI that can receive an access request to the anonymized customerdigital twin 5902. A requesting entity can use the API of the anonymizedcustomer digital twin 5902 to issue an access request. The accessrequest may be routed from the API to an access logic of the anonymizedcustomer twin 5902, which can determine if the requesting entity isentitled to access. In embodiments, users may monetize access toanonymized customer digital twins 5902, such as by subscription or anyother suitable monetization method.

The platform 604 may include, integrate, integrate with, manage,control, coordinate with, or otherwise handle enterprise customerengagement digital twins. Enterprise customer engagement digital twinsmay be a digital representation of a set of attributes of the enterprisecustomer that are relevant to engagement by the customer with a set ofofferings of an enterprise.

An environment for development of an enterprise customer engagementdigital twin may include a set of interfaces for developers in which adeveloper may configure an artificial intelligence system 1160 to takeinputs from selected data sources of the data storage layer 624 andevents or other data from the monitoring systems layer 614 and supplythem for inclusion in an enterprise customer engagement digital twin. Anenterprise customer engagement digital twin development environment maybe configured to take outputs and outcomes from various applications630. In embodiments, an enterprise customer engagement digital twin maybe provided for the wide range of value chain network applications 630mentioned throughout this disclosure and the documents incorporatedherein by reference.

Referring to FIG. 61 , the platform 604 may include, integrate,integrate with, manage, control, coordinate with, or otherwise handlecomponent digital twins 6002. Component digital twins 6002 may representevolving, continuously updated digital profiles of components 6002 ofvalue chain products 650. Component digital twins 6002 may allow formodeling, simulation, prediction, decision-making, classification, andthe like.

Product components can be depicted in a set of one or component digitaltwins 6002, such as by populating the component digital twins 6002 withvalue chain network data objects 1004, such as event data 1034, statedata 1140, or other data with respect to value chain network productcomponents.

A product 1510 may be any category of product, such as a finished good,software product, hardware product, component product, material, item ofequipment, consumer packaged good, consumer product, food product,beverage product, home product, business supply product, consumableproduct, pharmaceutical product, medical device product, technologyproduct, entertainment product, or any other type of product and/or setof related services, and which may, in embodiments, encompass anintelligent product 1510 that is enabled with a set of capabilities suchas, without limitation data processing, networking, sensing, autonomousoperation, intelligent agent, natural language processing, speechrecognition, voice recognition, touch interfaces, remote control,self-organization, self-healing, process automation, computation,artificial intelligence, analog or digital sensors, cameras, soundprocessing systems, data storage, data integration, and/or variousInternet of Things capabilities, among others. A component 6002 may beany category of product component.

As an example, a component digital twin 6002 may be populated withsupplier data, dimension data, material data, thermal data, price data,and the like.

A component digital twin 6002 may include a set of components,processes, services, interfaces, and other elements for development anddeployment of digital twin capabilities for visualization of value chainnetwork components 714 as well as for coordinated intelligence(including artificial intelligence system 1160, edge intelligence,analytics and other capabilities) and other value-added services andcapabilities that are enabled or facilitated with a component digitaltwin 6002.

In embodiments, the component digital twin 6002 may take advantage ofthe presence of multiple applications 630 within the value chainmanagement platform 604, such that a pair of applications may share datasources (such as in the data storage layer 624) and other inputs (suchas from the monitoring layer 614) that are collected with respect tovalue chain entities 652, as well sharing outputs, events, stateinformation and outputs, which collectively may provide a much richerenvironment for enriching content in a component digital twin 6002,including through use of artificial intelligence system 1160 (includingany of the various expert systems, artificial intelligence systems,neural networks, supervised learning systems, machine learning systems,deep learning systems, and other systems described throughout thisdisclosure and in the documents incorporated by reference) and throughuse of content collected by the monitoring layer 614 and data collectionsystems 640.

An environment for development of a component digital twin 6002 mayinclude a set of interfaces for developers in which a developer mayconfigure an artificial intelligence system 1160 to take inputs fromselected data sources of the data storage layer 624 and events or otherdata from the monitoring systems layer 614 and supply them for inclusionin a component digital twin 6002. A component digital twin developmentenvironment may be configured to take outputs and outcomes from variousapplications 630. In embodiments, a component digital twin 6002 may beprovided for the wide range of value chain network applications 630mentioned throughout this disclosure and the documents incorporatedherein by reference. In embodiments, a digital twin 650 may be generatedfrom other digital twins 1700. For example, a product digital twin 1780may be used to generate component digital twins 6002. In anotherexample, component digital twins 6002 may be used to generate productdigital twins 1780. In embodiments, a digital twin 1700 may be embeddedin another digital twin 1700. For example, a component digital twin 6002may be embedded in a product digital twin 1780 which may be embedded inan environment digital twin 6004.

In embodiments, a simulation management system 6110 may set up,provision, configure, and otherwise manage interactions and simulationsbetween and among digital twins 1700 representing value chain entities652.

In embodiments, the adaptive intelligent systems layer 614 is configuredto execute simulations in a simulation management system 6110 using thecomponent digital twins 6002 and/or digital twins 1700 of other valuechain network entities 652. For example, the adaptive intelligentsystems layer 614 may adjust one or more features of an environmentdigital twin 6004 as a set of component digital twins 6002 are subjectedto an environment. In embodiments, the adaptive intelligent systemslayer 614 may, for each set of features, execute a simulation based onthe set of features and may collect the simulation outcome dataresulting from the simulation.

For example, in executing a simulation on a set of component digitaltwins 6002 representing components of value chain product 1510 in anenvironment digital twin 6004, the adaptive intelligent systems layer614 can vary the properties of the environment digital twin 6110 and canexecute simulations that generate outcomes. During the simulation, theadaptive intelligent systems layer 614 may vary the environment digitaltwin temperature, pressure, lighting, and/or any other properties of theenvironment digital twin 6004. In this example, an outcome can be acondition of the component digital twin 6002 after being subjected to ahigh temperature. The outcomes from simulations can be used to trainmachine learning models 6120.

In embodiments, a machine learning system 6150 may receive training data6170, outcome data 6160, simulation data 6140, and/or data from othertypes of external data sources 6150 (weather data, stock market data,sports event data, news event data, and the like). In embodiments, thisdata may be provided to the machine-learning system 6150 via an API ofthe adaptive intelligent systems layer 614. In embodiments, the machinelearning system 6150 may receive simulation data 6140 relating to acomponent digital twin 6002 simulation. In this example, the simulationdata 6140 may be the properties of the component digital twins 6002 thatwere used to perform the simulation and the outcomes resultingtherefrom.

In embodiments, the machine learning system 6150 may train/reinforcemachine leaning models 6120 using the received data to improve themodels.

FIG. 62 illustrates an example of a risk management system 6102 thatinterfaces with the adaptive intelligent systems layer 614. In exampleembodiments, the risk management system 6102 may be configured to managerisk or liability with respect to a good or good component.

In embodiments, the machine-learning system 6150 trains one or moremodels 6120 that are utilized by the artificial intelligence system 1160to make classifications, predictions, and/or other decisions relating torisk management, including for products 650 and product components. Inembodiments, may be equipment components. In example embodiments, amodel 6120 is trained to mitigate risk and liability by detecting thecondition of a set of components. The machine-learning system 6150 maytrain the models using n-tuples that include the features pertaining tocomponents and one or more outcomes associated with the componentcondition. In this example, features for a component may include, butare not limited to, component material (plastic, glass, metal, or thelike), component history (manufacturing dates, usage history, repairhistory), component properties, component dimensions, component thermalproperties, component price, component supplier, and/or other relevantfeatures. In this example, outcomes may include whether the digital twinof the component 6002 is in operating condition. In this example, one ormore properties of the digital twins are varied for differentsimulations and the outcomes of each simulation may be recorded in atuple with the proprieties. Other examples of training risk managementmodels may include a model 6120 that is trained to optimize productsafety, a model that is trained to identify components with a highlikelihood of causing an undesired event, and the like.

In operation, the artificial intelligence system 1160 may use theabove-discussed models 6120 to make risk management decisions on behalfof a risk management system 6102 given one or more features relating toa task or event. For example, the artificial intelligence system 1160may determine the condition of a component. In this example, the riskmanagement system 6102 may provide the features of the component to theartificial intelligence system 1160. These features may includecomponent material, component history, component dimensions, componentcost, component thermal properties, component supplier, and the like. Inembodiments, the artificial intelligence system 1160 may feed thesefeatures into one or more of the models discussed above to obtain one ormore decisions. These decisions may include whether the component is inoperating condition.

In embodiments, the artificial intelligence system 1160 may leverage thecomponent digital twins 6002 to run simulations on the proposeddecisions.

The risk management system 6102 may then initiate a component resupplyevent using the decision(s) made by the artificial intelligence system1160. Furthermore, after the component resupply event, the outcomes ofthe event (e.g., improved product performance) may be reported to themachine-learning system 6150 to reinforce the models used to make thedecisions.

The platform 604 may include, integrate, integrate with, manage,control, coordinate with, or otherwise handle component attributedigital twins 6140. Component attribute digital twins 6140 may be adigital representation of a set of attributes of a set of supply chaincomponents in a supply for a set of products of an enterprise.

An environment for development of a component attribute digital twin6140 may include a set of interfaces for developers in which a developermay configure an artificial intelligence system 1160 to take inputs fromselected data sources of the data storage layer 624 and events or otherdata from the monitoring systems layer 614 and supply them for inclusionin a component attribute digital twin 6140. A component attributedigital twin development environment may be configured to take outputsand outcomes from various applications 630. In embodiments, a componentattribute digital twin 6140 may be provided for the wide range of valuechain network applications 630 mentioned throughout this disclosure andthe documents incorporated herein by reference.

In embodiments, the methods, systems and apparatuses include aninformation technology system having a value chain network managementplatform with an asset management application associated with maritimeassets and a data handling layer of the management platform includingdata sources containing information used to populate a training setbased on a set of maritime activities of one or more of the maritimeassets and one of design outcomes, parameters, and data associated withthe one or more maritime assets. The information technology system alsohas an artificial intelligence system that is configured to learn on thetraining set collected from the data sources, that simulates one or moreattributes of one or more of the maritime assets, and that generates oneor more sets of recommendations for a change in the one or moreattributes based on the training set collected from the data sources.The information technology system also has a digital twin systemincluded in the value chain network management platform that providesfor visualization of a digital twin of one or more of the maritimeassets including detail generated by the artificial intelligence systemof one or more of the attributes in combination with the one or moresets of recommendations.

Referring to FIG. 6 , the value chain network management platform 604orchestrates a variety of factors involved in planning, monitoring,controlling, and optimizing various entities and activities involved inthe value chain network 668 as it is applied to maritime assets,activities, logistics, and planning including supply and productionfactors, demand factors, logistics and distribution factors, and thelike. The management platform 604 can facilitate the monitoring andmanaging of supply factors and demand factors, the sharing of statusinformation about and between various entities as demand factors areunderstood and accounted for, as orders are generated and fulfilled, andas products are created and moved through a supply chain. Referring toFIG. 7 , the management platform 604 may include a set of value chainnetwork entities 652 including various delivery systems 632 that caninclude and connect to maritime facilities 622. The maritime facilities622 can include port infrastructure facilities 660, floating assets 620,and shipyards 638, and the like. In embodiments, the value chain networkmanagement platform 604 monitors, controls, and otherwise enablesmanagement (and in some cases autonomous or semi-autonomous behavior) ofa wide range of value chain network 668 processes, workflows,activities, events and applications 630 applicable in the maritimeenvironment.

Referring to FIGS. 6 and 11 , the management platform 604 deployed inthe maritime environment may include a set of data handling layers 608each of which is configured to provide a set of capabilities thatfacilitate development and deployment of intelligence, such as forfacilitating automation, machine learning, applications of artificialintelligence, intelligent transactions, state management, eventmanagement, process management, and many others, for a wide variety ofvalue chain network applications and end uses in the maritimeenvironment. In embodiments, the data handling layers 608 are configuredin a topology that facilitates shared data collection and distributionacross multiple applications and uses within the management platform 604by the value chain monitoring systems layer 614. The value chainmonitoring systems layer 614 may include, integrate with, and/orcooperate with various data collection and management systems 640,referred to for convenience in some cases as data collection systems640, for collecting and organizing data collected from or about valuechain entities 652, as well as data collected from or about the variousdata layers 624 or services or components thereof.

In embodiments, the data handling layers 608 are configured in atopology that facilitates shared or common data storage across multipleapplications and uses of the platform 604 by the value chainnetwork-oriented data storage systems layer 624, referred to herein forconvenience in some cases simply as the data storage layer 624 orstorage layer 624. For example, various data collected about the valuechain entities 652, as well as data produced by the other data handlinglayers 608, may be stored in the data storage layer 624, such that anyof the services, applications, programs, or the like of the various datahandling layers 608 can access a common data source, which may comprisea single logical data source that is distributed across disparatephysical and/or virtual storage locations. This may facilitate adramatic reduction in the amount of data storage required to handle theenormous amount of data produced by or about value chain networkentities 652 as applications 630 and uses of value chain networks growand proliferate. For example, a supply chain or inventory managementapplication in the value chain management platform 604, such as one forordering replacement parts for a machine or item of equipment, mayaccess the same data set about what parts have been replaced for a setof machines as a predictive maintenance application that is used topredict whether a component of a ship, or facility of a port is likelyto require replacement parts. Similarly, prediction may be used withrespect to resupply of items.

Referring to FIGS. 6 and 12 , the value chain network-oriented datastorage systems layer 624 may include, without limitation, physicalstorage systems, virtual storage systems, local storage systems 1190,distributed storage systems, databases, memory, network-based storage,network-attached storage systems. In embodiments, the storage layer 624may store data in one or more knowledge graphs in the graph databasearchitectures 1124, such as a directed acyclic graph, a data map, a datahierarchy, a data cluster including links and nodes, a self-organizingmap, or the like. In embodiments, the data storage layer 624 may storedata in a digital thread, ledger, distributed ledger or the like, suchas for maintaining a serial or other records of an entity 652 over time,including any of the entities described herein. In embodiments, thestorage layer 624 may include one or more blockchains 1180, such as onesthat store identity data, transaction data, historical interaction data,and the like, such as with access control that may be role-based or maybe based on credentials associated with a value chain entity 652, aservice, or one or more applications 630. Data stored by the datastorage systems 624 may include accounting and other financial data 730,access data 734, asset and facility data 1032, asset tag data 1178,worker data 1032, event data 1034, risk management data 732, pricingdata 738, safety data 664 and the like.

Referring to FIG. 8 , the value chain network management platform 604includes one or more sets of value chain entities 652 that may besubject to management by the management platform 604, may integrate withor into the management platform 604, and/or may supply inputs to and/ortake outputs from the management platform 604, such as ones involved inor for a wide range of value chain activities. These value chainentities 652 may include any of the wide variety of assets, systems,devices, machines, components, equipment, facilities, and individualsthat can support a wide range of operating facilities 712 includingmaritime facilities 622. Referring to FIGS. 63 , the maritime facilitiescan include port infrastructure facilities 7000. In embodiments, theport infrastructure facilities 7000 can include docks 7002, yards 7004,cranes 7008, roll-on/roll-off facilities 7010, ramps 7012, containers7014, container handling systems 7018, waterways 732, and locks 7020, asapplicable. In embodiments, the docks 7002 and their adjacent areas mayinclude piers 7022, basins 7024, stacking areas 7028, storage areas7030, and warehouses 7032. In embodiments, the container handlingsystems 7018 can include portainer tracking system and sensors 7040,such as for monitoring, reporting on, or managing one or more portainersor other systems for moving shipping containers, such as cranes (e.g.,Gottwald cranes, gantry cranes, and others), straddle carriers,multitrailers, reach stackers, and the like. In embodiments, the portinfrastructure facilities 7000 can further include gantry cranes 7042and the port vehicles 7044 that can be used to move containers 7014,such as straddle carriers. In embodiments, the port infrastructurefacilities 7000 also include refrigerated containers 7050 with dedicatedstacking areas 7052 and cooling infrastructure to maintain thecontrolled environments in the refrigerated containers 7050.

The port infrastructure facilities 7000 further include shipyardfacilities 638 and floating assets 620. The floating assets 620 caninclude ships 7060 and boats, container ships 7062, barges 7064,tugboats 7068, 7070, and dinghies 7072, as well as partially floatingassets, such as submarines, underwater drones, and the like. By way ofthese examples, the floating assets 620 can operate among facilities andother items at points of origin 610 and/or points of destination 628.The shipyard facilities 638 can include the hauling facilities 710 suchas many of the floating assets 620 as well as land-based vehicles andother delivery systems 632 used for conveying goods, such as trucks,trains, and the like

Referring to FIGS. 63 , orchestration of a set of deeply interconnectedvalue chain network entities 652 by the management platform 604 caninclude providing interconnectivity for the value chain network entities652 using local network connections, a peer-to-peer connections,connections through one or more mobile networks, and connections via acloud network facility, satellite uplinks, microwave communications orother connections. The management platform 604 may manage theconnections, configure or provision resources to enable connectivity,and/or manage applications 630 that take advantage of the connectionsknowing that are many maritime environments where connectivity may bepoor or non-existent relative to when the floating assets 620 are closerto port or other land-based communication systems. In many examples, aport infrastructure facility 660, such as a yard for holding shippingcontainers 7080, may inform a fleet of floating assets 620 viaconnections to the floating assets 620 that the port is near capacity.With this knowledge, the floating assets 620 movement can be varied toextend times including reducing approach speeds to delay arrival,direction to other ports, and the like. In further examples, the news ofthe port reaching capacity can result in starting a negotiation processwith the floating assets 620 looking to arrive at port. In embodiments,the negotiation process with the floating assets 620 can include anautomated negotiation based on a set of rules and governed by a smartcontract for the remaining capacity and enabling some floating assets620 to be redirected to alternative ports or holding facilities.

In embodiments, the maritime facilities 622 can include floating assets620 including many different ships 7060. Referring to FIGS. 64 and 65 ,the ship 7060 can be one or more container ships 7062 that can haul manyshipping containers 7080. In other examples, the ship 7060 can be one ormore container ships 7062 that can haul raw materials, processed goodsin bulk, gaseous cargo and many other forms of cargo not otherwisetransported in shipping containers 7080. In many examples, the ship 7060can include a bow area 7100. The bow area 7100 can include a bulbous bow7102. In some examples, the bulbous bow 7102 can be configured in-situin response to control from the management platform 604. Inboard fromthe bow area 7100 and traveling toward the stern area 7104 of the ship7060, the ship 7060 can include a forepeak tank 7110. In this same area,the ship 7060 can include one or more bow anchors 7112 and bow thrusters7114. Various passageways 7118 connect these areas in the bow area 7100.Depending on the configuration of the ship 7060, the hold 7120 can beconfigured and re-configured to accommodate various products such asproduct 1510, raw materials, material in process, and combinationsthereof. In some examples, the ship 7060 can include multiple holds7120. In examples, the container ship 7062 can be configured with eightholds: container hold 7130, 7132, 7134, 7138, 7140, 7142, 7144, and7148. Toward the stern area 7104, the ship 7060 includes an engine room7150 including one or more propulsion units 7152. Each of the one ormore propulsion units 7152 is fed by a fuel system 7154 and itsemissions are controlled by an exhaust system 7158. In various locationson the ship 7060, one or more fin stabilizers 7160 may be deployed. Inthe stern area 7104, the ship 7060 includes a steering gear area 7160below a rear deck area 7162. One or more rudders 7164 can extend fromthe steering gear area 7160.

One or more propellers 7170 can extend from the stern area 7104 with arotating power connection to the propulsion units. In embodiments, oneor more propellers 7170 can extend from the ship 7060 with an electricalconnection to the propulsion units but no physical rotating powerconnection. In embodiments, one or more propellers 7170 can extend fromthe ship 7060 with a hydraulic connection to the propulsion units but nophysical rotating power connection. In further examples, steam or otherworking fluids may be employed to drive the propulsion of the ship 7060.In further examples, mechanical rotating power, electrical drive,hydraulic drive, steam and various combinations thereof can be used forpropulsion. In various examples, the one or more propellers 7170 caninclude side propellers 7172 and a central propeller 7174. In otherexamples, two propellers 7170 can be deployed. In embodiments, thepropellers 7170 can be fixed such that the plane in which the propellerrotates is fixed relative to the ship 7060. By way of these examples,the propellers 7170 can be fixed and can be driven by mechanical linkageto propulsion units of the ship 7060. In other examples, the propellers7170 can be fixed and can be driven by electrical motors adjacent eachof the propellers 7170. In embodiments, the position of the propellers7170 can be variable such that the plane in which the propeller rotatesis movable relative to the ship 7060. By way of these examples, thepropellers 7170 can be driven by electrical motors adjacent to each ofthe propellers 7170. In one or more locations on the ship 7060, thepropellers 7170 can be deployed in pods that can include anindependently controlled and movable electrical drivetrain and propellerso that the entire pod can be moved into various positions to facilitateforward propulsion, steering, maneuvering, docking, evasive maneuvers,and the like.

In further examples, the ship 7060 is configured with one or moreballast tanks 7180. In various examples, the ship 7060 can include sideballast tanks 7182 and deep ballast tanks 7184. The ballast tanks 7180can each include pumping and draining systems 7190, cleaning systems7192, sensors 7194 to determine characteristics of the ballast watersuch as salinity, foreign particles, organic material, garbage,restricted content relative to geofenced areas, regulated zones, ad-hocdemarcated areas, and the like. The sensors 7194 can also determine tankcharacteristics including wear from fatigue, corrosion, physical damage,or the like. In the bow area 7100, the ship 7060 can include a windlass7200, a foremast 7202, and a crow's-nest 7204 on which various sensors7208 can be located to observe characteristics of the ship 7060, theweather and ambient conditions 7210, and navigational inputs 7212. Invarious locations on the ship 7060, one of more mooring winches 7220 canbe deployed to assist in docking, in connection to suitable mooringconnections points, connection other vessels in transit such as tenders,and the like. In various locations on the ship 7060, one or more hatchcovers 7222 can be deployed to permit access to various areas andpassageways on the ship 7060.

In further examples, the ship 7060 is configured as a container ship7062 that can be configured with eight holds: container hold 7130, 7132,7134, 7138, 7140, 7142, 7144, and 7148. In further examples, the ship7060 is configured as a container ship 7062 with various numbers ofholds 7120. In further examples, the ship 7060 is configured as acontainer ship 7062 with in-situ configurable holds. In furtherexamples, the ship 7060 is configured as a container ship 7062 withvarious numbers of holds some of which are in-situ configurable. Inembodiments, the holds 7120 can include one or more vents 7240 deployedto facilitate an atmosphere in the hold suitable for transit and for thecare of the cargo. In embodiments, the holds 7120 can include one ormore rigging and anchoring systems 7242 to secure one or more loadswithin holds 7120 configured or reconfigured for such cargo. Inembodiments, the holds 7120 can include one or more movable baffle anddunnage 7244 to secure one or more loads within holds 7120 configured orreconfigured for such cargo.

In further examples, the ship 7060 includes a wheelhouse 7250 and one ormore life rafts 7252 and lifeboats 7254. In further examples, the ship7060 includes nautical and satellite navigational equipment 7260. By wayof these examples, the ship can include direction finder antennae 7262,radar scanner 7264, a signal yard 7268. In these examples, the ship 7060includes a radar mast 7270 and a Suez signal light 7272, a funnel 7274and an antenna pole 7278.

In further examples, the ship 7060 includes one or more cranes 7280 thatcan be used to move things in and about the decks 7282 and in and out ofthe holds 7120 of the ship 7060. In these examples, the ship 7060 cancontain or carry on top many containers of various sizes includingtwenty-foot and forty-foot containers. In these examples, the ship 7060can contain or carry on top many containers of various sizes includingtwenty-foot dry freight containers, twenty-foot open-top containers,twenty-foot collapsible flat rack containers, twenty-foot refrigeratedcontainers, and the like. In these examples, the ship 7060 can containor carry on top many containers of various sizes including forty-foothigh cube containers, forty-foot open-top containers, forty-footcollapsible flat rack containers, forty-foot high cube refrigeratedcontainers, and the like. In these examples, the ship 7060 can containor carry on top many containers of various sizes includingforty-five-foot high cube dry containers, and the like.

In embodiments, the ship 7060 can contain engine units that include adiesel generator 7280 that can supply electrical power throughout theship 7060. The ship 7060 can also contain engine units that include acenter main diesel engine 7282 and one or more side main diesel engines7284. In embodiments, the ship 7060 can contain engine units that areconfigured to combust natural gas, propane, gasoline, methanol, and thelike. In embodiments, the ship 7060 can contain engine units that areconfigured to be powered by nuclear units that can be used to heat waterto steam-driven electrical systems. In embodiments, the ship 7060 cancontain engine units that are configured to be powered by nuclear unitsand internal combustion engines in a hybrid arrangement. In embodiments,the ship 7060 can contain engine units that are configured to be poweredby nuclear units and internal combustion engines, and other renewablesin a hybrid arrangement such as solar and wind where each of these canfeed an electrical and battery system to power propulsion and shipoperations.

In embodiments, the ship 7060 can contain multiple bulkheads 7290. Byway of these examples, the engine room can be framed in engine roombulkheads 7292 to contain the various powerplant units. In embodiments,the cargo and hold region of the ship 7060 can contain hold bulkheads7294 to contain the various powerplant units. In embodiments, the ship7060 can contain structural transverse bulkheads 7300 and axialbulkheads 7302.

In embodiments, the maritime facilities 622 can include floating assets620 including many different barges 7500. Referring to FIG. 66 , one ormore of the barges 7500 can be transport barges, cargo barges,submersible barges, and the like that can in size and capacity. In manyexamples, barges are available in many varieties of towed barges andself-propelled ships including submersible heavy lift vessels. In manyexamples, the barges 7500 can be towed or pushed by tug boats 7510 totransport from one location to another. In many examples, the barges7500 can be flat top and bottom and can be equipped with navigationallights 7520, fairleads 7522 and towing points 7524.

In some examples, the barges 7500 can be designed to be submerged so asto pick up cargoes 7530 such as floating cargoes. By way of theseexamples, the barges 7500 can be equipped with a forecastle 7540 and adeck structure 7542 at a bow area 7550 opposite a deck structure 7544 ata stern area 7552. There can be additional deck structure 7548 betweenthe bow area 7550 and the stern area 7552 that can be configured andre-configured to hold the cargoes 7530. In these examples, the barges7500 can be equipped with their own ballast system 7560. In embodiments,the barges 7500 can include a modular steel box 7570 and stabilitycasings 7572 that may be added at the stern area 7552 to somepredetermined degree to effectively provide additional portions of ahull 7580 in the water 7582 that can be shown to enhance the stabilityof the barge 7500 and its cargoes 7530 as the deck structures 7542,7544, 7548 go through a waterline 7584. In these examples, the modularsteel box 7570 and stability casings 7572 can be removable and can bestowed away on one of the deck structures 7542, 7544, 7548 of the barge7500 or stored onshore when not required. In doing so, the barge 7500can be relatively more efficient when lighter loads warrant therelatively smaller hull structure.

In many examples, barges 7500 can be classified not only by their lengthand width but also how they are used, launched and the like. In someexamples, one or more of the barges 7500 can be less than 200 feet inlength and 50 feet wide. By way of these examples, the barge 7500 caninclude small pontoons can be used for carrying small structures insheltered inshore waters. In some examples, one or more of the barges7500 can be about 250 feet by 70 feet and can include small pontoons tosupport the barge 7500 that is otherwise configured without an onboardballast system. By way of these examples, barges in these configurationscan be used to transport small offshore loads, do work in and near portinfrastructures, perform maintenance in a shipyard, etc. In someexamples, one or more of the barges 7500 can be about 300 feet and canbe 90 or 100 feet wide. By way of these examples, one or more barges inthese configurations can be used as standard cargo barges but may not beequipped with an onboard ballast system. In some examples, one or morebarges 7500 can be about 400 feet by 100 feet and these barges can beequipped with an onboard ballast system.

In some examples, one or more of the barges 7500 can be about 450 feetand longer and can be deployed with an onboard ballasting systems 7590.By way of these examples, one or more of the barges 7500 can also bedeployed with skid beams 7592. One or more of the barges 7500 can alsobe deployed with rocker arms 7594 at the stern area 7552 to enable, forexample, the launching of jackets or other loads that may be too heavyto lift. In examples, the Heerema H851 brand barge is nominally 850 feetlong by 200 feet wide and can be a suitable example of one of thelargest commercially available barges.

In embodiments, one or more of the barges 7500 can also be configured asa submersible barge 7600, which can be a towed barge that can beequipped with stability casings 7602 in the stern area 7552. Inexamples, the submersible barge 7600 can be configured with a ship-likebow structure 7604. In these examples, the ship like bow structure 7604can be configured with a bridge 7608 sufficiently tall to enable thesubmerging of the barge above at least a portion of its deck structures.In examples, the Boa brand barges have nominal dimensions of 400 feet by100 feet, the AMT brand barges have nominal dimensions 470 feet by 120feet and Hyundai brand barges having nominal dimensions 460 feet by 120feet can be suitable examples of commercially available submersiblebarges. By way of these examples, these barges can submerge up to 18 to24 feet above their decks.

It will be appreciated in light of the disclosure that barges are ratedand paired with jobs in terms of deadweight which provides a broadindication of the barges' carrying capacity. The barges, however, haveadditional requirements such as their global strength, local deck andframe strengths and height of the cargo's center of gravity. With regardto center of gravity, one exemplary barge may be able to transport a20,000-ton structure with its center of gravity very close to the decksufficiently tied and supported on the deck. The same exemplary bargemay only be able to transport a half of the weight if the cargo has arelatively high center of gravity. With that in mind, many attributes ofone or more of the barges are the placement, orientation, center ofgravity and weight of the cargoes on their decks.

In embodiments, one of the barges can be towed by one of the ships,tugboats 7510, or the like with a towing bridle 7610. In many examples,two lines 7612 can run from tow brackets 7614 through fairleads 7618 onone of the barges and connect to a triplate 7620 on the barge throughtowing shackles 7622. By way of this example, a third line 7630 canconnect the triplate 7620 to a winch 7640 on one of the tugboats 7510.In further examples, an emergency wire 7642 can be installed along thelength of the barge. The emergency wire 7642 can be attached to aconnector 7644 that can terminate with a buoy 7650. The buoy 7650 cantrail behind the barge 7650 during tow and can form part of the towingarrangement.

In some examples, roll accelerations of the barge can be directlyproportional to the transverse stiffness of the barge, which can bemeasured by its metacentric height. In some arrangements, a barge canhave a large metacentric height and as a result, roll accelerations canbe severe. In further examples with relatively tall cargo, themetacentric height can be low resulting in the period and amplitude ofroll and the static force resulting from the load being greater but thedynamic component may be less. In many examples, attributes of the barge7500 include positioning of cargoes 7530 on its deck structures and itseffective metacentric height. In further examples, counter-rollmechanisms 7660 can be installed on the barge 7500. By way of theseexamples, the adaptive intelligence layer 614 can update the program ofthe counter-roll mechanisms 7660 and can be shown to increase itsefficacy to changing cargo load and water and weather conditions. Inembodiments, the adaptive intelligence layer 614 can update the speedand angles of the of the counter-roll mechanisms 7660 and can be shownto increase its efficacy to changing cargo load and water and weatherconditions.

In embodiments, the management platform 604 may include a set of valuechain network entities 652 including various delivery systems 632 thatcan include and connect to the maritime facilities 622. The maritimefacilities 622 can include port infrastructure facilities 660, floatingassets 620, and shipyards 638, and the like. In embodiments, the valuechain network management platform 604 monitors, controls, and otherwiseenables management (and in some cases autonomous or semi-autonomousbehavior) of a wide range of value chain network 668 processes,workflows, activities, events and applications 630 applicable in themaritime environment.

The maritime facilities 622 can include one or more ships 7060 ofvarious sizes to service the facilities. The maritime facilities 622 caninclude one or more fixed or moored navigation aids within the water oron land to facilitate the movement ships of various sizes and vehicleson land. In embodiments, the maritime facilities 622 can be configuredas a seaport in that it can be configured to accept deep-draft shipswith a draft of 20 feet or more. In embodiments, some of the largermaritime facilities 622 can include areas outside the boundaries of theseaports, shipyard, maritime ports, and the like that are related toport operations or to an intermodal connection to the seaports,shipyard, maritime ports, and the like.

In embodiments, the management platform 604 can manage port gate-in andgate-out improvements to the logistics of the flow of assets and cargoesaround the maritime facilities 622. In embodiments, the managementplatform 604 can manage road improvements both within and connecting tothe maritime facilities 622. In embodiments, the management platform 604can manage rail improvements both within and connecting to the maritimefacilities 622. In embodiments, the management platform 604 can manageberth improvements in the maritime facilities 622 including to docks,wharves, piers and the like. In embodiments, the management platform 604can manage berth improvements including dredging at the berths, approachand departure areas adjacent to the berth, and in areas around maritimefacilities. In embodiments, the management platform 604 can manage cargomoving equipment used on land. In embodiments, the management platform604 can manage facilities necessary to improve cargo transport includingsilos, elevators, conveyors, container terminals, roll-on/roll-offfacilities including parking garages necessary for intermodal freighttransfer, warehouses including refrigerated facilities, bunkeringfacilities for oil or gas products, lay-down areas, transit sheds, andthe like. In embodiments, the management platform 604 can manageutilities necessary for standard operations including lighting,stormwater, and the like that can be incidental to a larger set ofmaritime facilities. In embodiments, the management platform 604 canmanage port-related intelligent transportation system hardware andsoftware including all technologies used to promote efficient portmovements including routing and communications for vessels, trucks, andrail cargo movements as well as flow-through processing forimport/export requirements, storage and tracking, and asset/equipmentmanagement. In embodiments, the management platform 604 can managephytosanitary treatment facilities to support phytosanitary treatmentrequirements. In embodiments, the management platform 604 can manage,configure and re-configure fully automated cargo-handling equipment.

In embodiments, the adaptive intelligent systems layer 614 may include aset of systems, components, services and other capabilities thatcollectively facilitate the coordinated development and deployment ofintelligent systems, such as ones that can enhance one or more of theapplications 630 at the application platform 604; ones that can improvethe performance of one or more of the components, or the overallperformance (e.g., speed/latency, reliability, quality of service, costreduction, or other factors) of the connectivity facilities 642; onesthat can improve other capabilities within the adaptive intelligentsystems layer 614; ones that improve the performance (e.g.,speed/latency, energy utilization, storage capacity, storage efficiency,reliability, security, or the like) of one or more of the components, orthe overall performance, of the value chain network-oriented datastorage systems 624; ones that optimize control, automation, or one ormore performance characteristics of one or more value chain networkentities 652; or ones that generally improve any of the process andapplication outputs and outcomes 1040 pursued by use of the platform604.

These adaptive intelligent systems 614 may be deployed in and among themaritime facilities 622 and floating assets 620. These adaptiveintelligent systems 614 may include a robotic process automation system1442, a set of protocol adaptors 1110, a packet acceleration system1410, an edge intelligence system 1430 (which may be a self-adaptivesystem), an adaptive networking system 1430, a set of state and eventmanagers 1450, a set of opportunity miners 1460, a set of artificialintelligence systems 1160, a set of digital twin systems 1700, a set ofentity interaction management systems 1902 (such as for setting up,provisioning, configuring and otherwise managing sets of interactionsbetween and among sets of value chain network entities 652 in the valuechain network 668), and other systems.

In embodiments, a set of digital twin systems 1700 may be deployed foreach of the maritime facilities 622 and each of the floating assets 620.Referring to FIG. 6 , the connected value chain network 668 benefitsfrom digital twin systems deployed throughout the value chain networkmanagement platform 604 to facilitate the management, visualization, andmodeling of the orchestration of a variety of factors involved inplanning, monitoring, controlling, and optimizing various entities andactivities involved in the value chain network 668, such as supply andproduction factors, demand factors, logistics and distribution factors,and the like. By virtue of the unified platform 604 for monitoring andmanaging supply factors and demand factors, digital twins for statusinformation can be shared about and between various entities tofacilitate modeling and analytics and to provide for visualization ofchanging demand factors becomes operational realities, as orders aregenerated and fulfilled, and as products are created and moved through asupply chain.

In embodiments, the value chain monitoring systems layer 614 and itsdata collection systems 640 may include a wide range of systems for thecollection of data from the maritime facilities 622 and the floatingassets 620. This layer may include, without limitation, real timemonitoring systems 1520 (such as onboard monitoring systems like eventand status reporting systems on ships and other floating assets, ondelivery vehicles, on trucks and other hauling assets, and in shipyards,ports, warehouses, distribution centers and other locations; on-boarddiagnostic (OBD) and telematics systems on floating assets, vehicles andequipment; systems providing diagnostic codes and events via an eventbus, communication port, or other communication system; monitoringinfrastructure (such as cameras, motion sensors, beacons, RFID systems,smart lighting systems, satellite connections, asset tracking systems,person tracking systems, and ambient sensing systems located in variousenvironments where value chain activities and other events take place),as well as removable and replaceable monitoring systems on maritimeassets and cargo or other assets contained therein or in transitthereon, such as portable and mobile data collectors, RFID and other tagreaders, smart phones, tablets and other mobile devices that are capableof data collection and the like); software interaction observationsystems 1500 that can be deployed into portable and onboard systems ofthe maritime facilities 622 and floating assets 620; visual monitoringsystems 1930 such as using video and still imaging systems, LIDAR, IRand other systems that allow visualization of items, people, materials,components, machines, equipment, personnel, and the like to detail cargoin the hold of floating assets 620, to detail activity of personal andgear deployed at the maritime facilities 622 and on the floating assets620; point of interaction systems (such as dashboards, user interfaces,and control systems for value chain entities); physical processobservation systems 1510 (such as for tracking physical activities ofoperators, workers, customers, or the like, physical activities ofindividuals (such as shippers, delivery workers, packers, pickers,assembly personnel, customers, merchants, vendors, distributors andothers), physical interactions of workers with other workers,interactions of workers with physical entities like machines andequipment, and interactions of physical entities with other physicalentities, including, without limitation, by use of video and still imagecameras, motion sensing systems (such as including optical sensors,LIDAR, IR and other sensor sets), robotic motion tracking systems (suchas tracking movements of systems attached to a human or a physicalentity) and many others; machine state monitoring systems 1940(including onboard monitors and external monitors of conditions, states,operating parameters, or other measures of the condition of any valuechain entity, such as a machine or component thereof, such as a machine,such as a client, a server, a cloud resource, a control system, adisplay screen, a sensor, a camera, a vehicle, a robot, or othermachine); sensors and cameras 1950 and other IoT data collection systems1172 (including onboard sensors, sensors or other data collectors(including click tracking sensors) in or about a value chain environment(such as, without limitation, a point of origin, a loading or unloadingdock, a vehicle or floating asset used to convey goods, a container, aport, a distribution center, a storage facility, a warehouse, a deliveryvehicle, and a point of destination), cameras for monitoring an entireenvironment, dedicated cameras for a particular machine, process,worker, or the like, wearable cameras, portable cameras, camerasdisposed on mobile robots, cameras of portable devices like smart phonesand tablets, and many others, including any of the many sensor typesdisclosed throughout this disclosure or in the documents incorporatedherein by reference); indoor location monitoring systems 1532 (includingcameras, IR systems, motion-detection systems, beacons, RFID readers,smart lighting systems, triangulation systems, RF and other spectrumdetection systems, time-of-flight systems, chemical noses and otherchemical sensor sets, as well as other sensors); user feedback systems1534 (including survey systems, touch pads, voice-based feedbacksystems, rating systems, expression monitoring systems, affectmonitoring systems, gesture monitoring systems, and others); behavioralmonitoring systems 1538 (such as for monitoring movements, shoppingbehavior, buying behavior, clicking behavior, behavior indicating fraudor deception, user interface interactions, product return behavior,behavior indicative of interest, attention, boredom or the like,mood-indicating behavior (such as fidgeting, staying still, movingcloser, or changing posture) and many others); and any of a wide varietyof Internet of Things (IoT) data collectors 1172, such as thosedescribed throughout this disclosure and in the documents incorporatedby reference herein.

Referring to FIG. 26 , a set of opportunity miners 1460 may be providedas part of the adaptive intelligence layer 614, which may be configuredto seek and recommend opportunities to improve one or more of theelements of the platform 604, such as via addition of artificialintelligence 1160, automation (including robotic process automation1402), or the like to one or more of the maritime facilities 622 and foreach of floating assets 620 including their systems, sub-systems,components, applications with which the platform 100 interacts. Inembodiments, the opportunity miners 1460 may be configured or used bydevelopers of AI or RPA solutions to find opportunities for bettersolutions and to optimize existing solutions in a value chain network668. In embodiments, the opportunity miners 1460 may include a set ofsystems that collect information within the management platform 604 andcollect information within, about and for a set of maritime facilities622 and for each of floating assets 620, where the collected informationhas the potential to help identify and prioritize opportunities forincreased automation and/or intelligence about the value chain network668, about applications 630, one or more of the maritime facilities 622and the floating assets 620. For example, the opportunity miners 1460may include systems that observe clusters of value chain network workersby time, by type, and by location (whether on the water or land), suchas using cameras, wearables, or other sensors, such as to identifylabor-intensive areas and processes in set of value chain network 668environments. These may be presented, such as in a ranked or prioritizedlist, or in a visualization (such as a heat map showing dwell times ofcustomers, workers or other individuals on a map of an environment or aheat map showing routes traveled by customers or workers within anenvironment) to show places with high labor activity. In embodiments,analytics 838 may be used to identify which environments or activitieswould most benefit from automation for purposes of improved deliverytimes, mitigation of congestion, and other performance improvements.

In embodiments, opportunity mining may include facilities forsolicitation of appropriate training data sets that may be used tofacilitate process automation. For example, certain kinds of inputs, ifavailable, would provide very high value for automation, such as videodata sets that capture very experienced and/or highly expert workersperforming complex tasks. This information becomes even more valuablewhen collected in close proximity to other maritime facilities 622 andwith deployed floating assets 620. Opportunity miners 1460 may searchfor such video data sets as described herein; however, in the absence ofsuccess (or to supplement available data), the management platform 604may include systems by which a user at a maritime facility or deployedon a maritime asset may specify a desired type of data, such as softwareinteraction data (such as of an expert working with a program to performa particular task), video data (such as video showing a set of expertsperforming a certain kind of delivery process, unloading process,securing and logistics process, cleaning and maintenance process, acontainer movement process, or the like), and/or physical processobservation data (such as video, sensor data, or the like). Theresulting library of interactions captured in response to thespecification may be captured as a data set in the data storage layer624, such as for consumption by various applications 630, adaptiveintelligence systems 614, and other processes and systems. Inembodiments, the library may include videos that are specificallydeveloped as instructional videos, such as to facilitate developing anautomation map that can follow instructions in the video, such asproviding a sequence of steps according to a procedure or protocol,breaking down the procedure or protocol into sub-steps that arecandidates for automation, and the like. In embodiments, such videos maybe processed by natural language processing, such as to automaticallydevelop a sequence of labeled instructions that can be used by adeveloper to facilitate a map, a graph, or other models of a processthat assists with development of automation for the process.

In embodiments, the value chain monitoring systems layer 614 and itsdata collection systems 640 may include an entity discovery system 1900for discovering one or more value chain network entities 652, such asany of the entities described throughout this disclosure and especiallythose that can be loaded and offloaded as control passes from variousmaritime facilities 622 and floating assets 620. This may includecomponents or sub-systems for searching for entities at maritimefacilities 622 and floating assets 620 within the value chain network668, such as by device identifier, by network location, by geolocation(such as by geofence), by indoor location (such as by proximity to knownresources, such as IoT-enabled devices and infrastructure, Wifi routers,switches, or the like), by cellular location (such as by proximity tocellular towers), by maritime navigation aids and vessel identitybeacons, by identity management systems (such as where an entity 652 isassociated with another entity 652, such as an owner, operator, user, orenterprise by an identifier that is assigned by and/or managed by theplatform 604), and the like. In these examples, an entity discoverysystem 1900 may interact with established maritime asset logisticsystems used to track traffic and location. In these examples, an entitydiscovery system 1900 may interact with established maritime assetautopilot and auto-navigation systems obtaining information relevant tointended navigation destinations and from there, the error and magnitudeof corrective action need to arrive at the navigation destination.

Referring to FIG. 22 , the adaptive intelligence layer 614 may include avalue chain network digital twin system 1700, which may include a set ofcomponents, processes, services, interfaces and other elements fordevelopment and deployment of digital twin capabilities forvisualization of various value chain entities 652 in environments, andapplications 630, as well as for coordinated intelligence (includingartificial intelligence 1160, edge intelligence 1420, analytics andother capabilities) and other value-added services and capabilities thatare enabled or facilitated with a digital twin 1700. In embodiments, adigital twin system 1700 may be deployed with each facility (or groupsthereof) among the maritime facilities 622 and may be deployed for eachof floating assets 620. In many instances, each floating asset 620 andphysical assets in the maritime facilities 622 can be coordinated andmanaged with its digital twin supported by the digital twin system 1700.Without limitation, a digital twin system 1700 may be used for and/orapplied to each of the processes that is managed, controlled, ormediated by each of the set of applications 614 of the platformapplication layer that may be deployed in various systems, networks, andinfrastructures (or across groups thereof) of the floating assets 620and in and among the maritime facilities 622.

In embodiments, the digital twin 1700 may take advantage of the presenceof multiple applications 630 within the value chain management platform604, such that a pair of applications may share data sources (such as inthe data storage layer 624) and other inputs (such as from themonitoring layer 614) that are collected (to support fusion of collectedsignals and the like) with respect to value chain entities 652, as wellsharing outputs, events, state information and outputs, whichcollectively may provide a much richer environment for enriching contentin a digital twin 1700, including through use of artificial intelligence1160 including any of the various expert systems, artificialintelligence systems, neural networks, supervised learning systems,machine learning systems, deep learning systems, and other systemsdescribed throughout this disclosure and in the documents incorporatedby reference and through use of content collected by the monitoringlayer 614 and data collection systems 640.

Referring to FIG. 23 , any of the value chain network entities 652 canbe depicted in a set of one or more digital twins 1700, such as bypopulating the digital twin 1700 with value chain network data object1004, such as event data 1034, state data 1140, or other data withrespect to value chain network entities 652, applications 630, orcomponents or elements of the platform 604 as described throughout thisdisclosure.

Thus, the platform 604 may include, integrate, integrate with, manage,control, coordinate with, or otherwise handle any of a wide variety ofdigital twins 1700, such as distribution twins 1714 (such asrepresenting distribution facilities, assets, objects, workers, or thelike); warehousing twins 1712 (such as representing warehousefacilities, assets, objects, workers and the like); port infrastructuretwins 1714 (such as representing a seaport, an airport, or otherfacility, as well as assets, objects, workers and the like); shippingfacility twins 1720; operating facility twins 1172; customer twins 1730;worker twins 1740; wearable/portable device twins 1750; process twins1760; machine twins 21010 (such as for various machines used to supporta value chain network 668); product twins 1780; point of origin twins1502; supplier twins 1630; supply factor twins 1650; maritime facilitytwins 1572; floating asset twins 1570; shipyard twins 1620; destinationtwins 1562; fulfillment twins 1600; delivery system twins 1610; demandfactor twins 1640; retailer twins 1790; ecommerce and online site andoperator twins 1800; waterway twins 1810; roadway twins 1820; railwaytwins 1830; air facility twins 1840 (such as twins of aircraft, runways,airports, hangars, warehouses, air travel routes, refueling facilitiesand other assets, objects, workers and the like used in connection withair transport of products 650); autonomous vehicle twins 1850; roboticstwins 1860; drone twins 1870; and logistics factor twins 1880; amongothers.

Referring to FIG. 27 , additional details of an embodiment of theplatform 604 are provided, in particular relating to elements of theadaptive intelligence layer 614 that facilitate improved edgeintelligence, including the adaptive edge compute management system 1400and the edge intelligence system 1420. These elements provide a set ofsystems that adaptively manage “edge” computation, storage andprocessing, such as by varying storage locations for data and processinglocations (e.g., optimized by AI) between on-device storage, localsystems, peer-to-peer, in the network and in the cloud. These elementscan enable facilitation of a dynamic definition by a user, such as adeveloper, operator, or host of the platform 102, of what constitutesthe “edge” for purposes of a given application anywhere in the world andespecially in regions of the oceans where connectivity can beconstrained. For example, for environments where data connections areslow or unreliable (such as where a facility does not have good accessto cellular networks (such as due to remoteness on the globe), shieldingor interference (such as where density of network-using systems, thickmetals hulls of container ships, thick metal container walls, underwateror underground location, or presence of large metal objects (such asvaults, hulls, containers, cranes, stacked raw materials, and the like)interferes with networking performance), and/or congestion (such aswhere there are many devices seeking access to limited networkingfacilities), edge computing capabilities can be defined and deployed tooperate on the local area network of an environment, in peer-to-peernetworks of devices, or on computing capabilities of local value chainentities 652. Where strong data connections are available (such as wheregood backhaul facilities exist), edge computing capabilities can bedisposed in the network, such as for caching frequently used data atlocations that improve input/output performance, reduce latency, or thelike. Thus, adaptive definition and specification of where edgecomputing operations are enabled, under control of a developer oroperator, or optionally determined automatically among a fleet ordeployed in a geographic region, such as by an expert system orautomation system that may be based on detected network conditions foran environment. In embodiments, edge intelligence 1420 enablesadaptation of edge computation (including where computation occurswithin various available networking resources, how networking occurs(such as by protocol selection), where data storage occurs, and thelike) that is multi-application aware, such as accounting for QoS,latency requirements, congestion, and cost as understood and prioritizedbased on awareness of the requirements, the prioritization, and thevalue of edge computation capabilities across more than one application.

In embodiments, the digital twin system 1700 may host floating assettwins 1570 that can be associated with one or more of the floatingassets 620. By way of these examples, one or more of the floating assettwins 1570 can simulate how one or more of the floating assets 620 willperform without needing to test the one or more of the floating assets620 in the real world. Further examples include visualization of allsystems of the ship, its navigation course, and functional needsincluding various details all forms of information on a ship, fromengine performance to hull integrity, available at a glance throughoutthe full lifetime of the vessel through its floating asset twins 1570.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide beneficial visualization of any and allimportant components of the one or more the floating assets 620. The useof the floating asset twins 1570 during operation can be shown to bebeneficial to carry out analyses and improve the operation on thestructural and functional components of the floating assets 620. Infurther examples, use of the floating asset twins 1570 during operationof the one or more of the floating assets 620 can be used to modelin-situ hydrodynamic and aerodynamic changes to the structures and hullsurfaces of the floating assets 620. In embodiments, the floating assets620 can deploy systems to alter the configuration of the cross-sectionsof certain portions of the hull, alter the configuration of hydrodynamiccontrol surfaces below the water line, alter the configuration ofaerodynamic control surfaces above the waterline, extended additionalbuoyant members from the hull to improve hull stability during certainmaneuvers, and the like. In these examples, artificial intelligencesystems 1160 can study simulated hull configurations deployed on thefloating asset twins 1570 to determine a schedule of hull configurationchanges to improve fuel efficiency using known routes of travel andhistorical weather patterns.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to benefit operators as they can plan for more efficientinspections and maintenance of one or more floating assets 620. Inembodiments, use of the port infrastructure twins 1714 during operationcan be shown to benefit operators that can plan for more efficientinspections and maintenance of one or more physical assets in themaritime facilities 622. This can also lead to an extension of thephysical assets' lifetimes, as preventive measures will be taken toavoid damages.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide operators with an ability to create visualmodels of the ship and its underlying systems, such as engine spaces andpumps, and continuously record its fuel consumption, distributed onsources of energy, such as engines, boilers and batteries. By way ofthese examples, operators can plan for more efficient operations,inspections and maintenance of one or more floating assets 620. Inembodiments, use of the port infrastructure twins 1714 during operationcan be shown to provide operators with ability to create visual modelsof the maritime assets at a port, on land, moored in location and placedas navigation aids including their underlying systems, such as systemspowerplants, and continuously record their energy consumption,distributed on sources of energy, such as engines, boilers andbatteries. By way of these examples, operators can plan for moreefficient operations, inspections and maintenance of one or morephysical assets in the maritime facilities 622. In embodiments, thedigital twin systems can include simulation and analytical models thatcan be developed to acquire the optimum fuel consumption for aparticular voyage with a specific cargo, by including external factorssuch as wind, current and weather conditions. In embodiments, thedigital twin systems can include simulation and analytical models thatcan be developed to acquire the optimum energy consumption for aparticular port activity such as unloading with a specific cargo, byincluding external factors such as weather conditions and other assetsmonitored by the adaptive intelligence layer 614.

In embodiments, use of the floating asset twins 1570 and the portinfrastructure twins 1714 during operation can be shown to provideoperators with ability to visualize control and adapt the operation ofmachinery systems in one or more floating assets 620 or deployed in thephysical assets in the maritime facilities 622, especially when thesupply chain is across the one or more floating assets 620 and thephysical assets in the maritime facilities 622 and processes can beheld, increased, decreased based on the progress of other processed onland or on the water.

In embodiments, use of the floating asset twins 1570 and the portinfrastructure twins 1714 during operation can be shown to provideoptimal points during the voyage or during service life on land toretrofit batteries and replace other switchgear. In embodiments, use ofthe floating asset twins 1570 during operation can be shown to provide abasis for changing to more powerful, more efficient, or more versatileengines, thrusters or other propulsion systems upon the usualmaintenance cycles or at opportune times for retrofit of components.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide a basis for tuning a schedule to adjust thefront bulbous bow of the floating assets 620 to improve efficient flowaround the bow of the vessel in various combinations of vessel speed,water activity and weather. In these examples, the front bulbous bow canadjust its shape based on the predetermined schedule or the revisedschedule adjust by the adaptive intelligence layer 614 for a shape ofthe bow for most efficient running.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide optimal points during the voyage to perform hullcleaning, maintenance or painting or perform propeller cleaning,maintenance or replacement. In embodiments, use of the floating assettwins 1570 during operation can be shown to provide basis for schedulingwhen hull or propeller cleaning is needed, where in the journeycontributes to greatest need to clean systems and determining withsimulation using the floating asset twins 1570 whether such maintenancejustified or routing of the floating assets 620 to different passagesmay inflict less of a maintenance burden.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide detailed simulation and visualization of optimalpoints during the voyage to perform hull cleaning, maintenance orpainting or perform propeller cleaning, maintenance or replacement. Inembodiments, use of the floating asset twins 1570 during operation canbe shown to provide basis for scheduling when hull or propeller cleaningis needed, where in the journey contributes to greatest need to cleansystems and determining with simulation using the floating asset twins1570 whether such maintenance justified or routing of the floatingassets 620 to different passages may inflict less of a maintenanceburden.

In embodiments, use of the floating asset twins 1570 during operationcan be shown to provide detailed simulation and visualization theperformance of one or more ships or floating assets 620 on a detailedlevel so users can see the effects of design choices and changes on theone or more ships or floating assets 620 as they simulate historicalvoyages, predicted voyages, and previous voyages modified to furthersimulate activity encountered to enhance training and safety. Inembodiments, use of the floating asset twins 1570 during operation canbe shown to provide detailed simulation and visualization theperformance of multiple ships or floating assets 620 on a detailed levelso users can make use of the digital twins for benchmarking performancetowards the other ships or maritime assets and these comparisons can beused to simulate historical voyages, predicted voyages, and previousvoyages modified to further simulate activity encountered to enhancetraining and safety.

In embodiments, use of the floating asset twins 1570 can be shown toprovide ship owners a tool for visualization of ships and theirsubsystems (and various other maritime assets), qualification andanalytics of operational data, optimization of ship performance,improved internal and external communication, safe handling of increasedlevels of autonomy and safe decommissioning.

In embodiments, use of the floating asset twins 1570 can be shown toprovide equipment manufacturers a tool to facilitate system integration,demonstrate technology performance, perform system quality assurance andpromote additional services for monitoring and maintenance.

In embodiments, use of the floating asset twins 1570 and the portinfrastructure twins 1714 can be shown to provide authorities asystematic framework that can be set up with applications to feed liveinformation and generate required reports from each maritime assetwhether ships, barges, other floating assets, and port infrastructureincluding moored navigation aids, cargo in unloaded and loadedconditions and even personnel that move throughout the portinfrastructure to ensure its operation. In many examples, use of thefloating asset twins 1570 and the port infrastructure twins 1714 can beshown to ensure higher quality reporting on critical issues withoutputting additional burdens or cognitive load on crew already ensuringoperations of the various maritime assets. In many examples, use of thefloating asset twins 1570 and the port infrastructure twins 1714 can beshown to ensure higher quality reporting on legal and regulatory issuesby providing time-stamped ledgers of activity paired with agreements andcontracts underlying the commerce supporting the maritime activitywithout putting additional burdens or cognitive load on crew alreadyensuring operations of the various maritime assets.

In embodiments, use of the floating asset twins 1570 and the portinfrastructure twins 1714 can be shown to provide universities,colleges, and municipalities with platforms on which to increase systemunderstanding and facilitate knowledge exchange enhancing research anddevelopment and education in a range of technological disciplines. Byway of these examples, use of the floating asset twins 1570 and the portinfrastructure twins 1714 can be shown to provide maritime academiesplatforms for training that can increase the candidates' understandingof the whole ship or specific maritime asset and train them in systemsunderstanding to see the integrated consequences of actions taken as itaffects that asset, all (or some) of the assets including floating andinfrastructure assets. In these examples, systems understanding can beshown to be improved because the integrated consequences of actionstaken can be seen at the asset level, the fleet of asset level, theinfrastructure level, and the business level showing how activity infleet can affect the profitability of the fleet with combinations ofimproving revenues and reducing expense where it makes sense all ofwhich can be visualized and interpreted from the floating asset twins1570 and the port infrastructure twins 1714 including suggestions fromthe adaptive intelligence layer 614.

In embodiments, an information technology system including a value chainnetwork management platform 604 can have an asset management application814 such as a maritime fleet management application 880 associated withone or more maritime assets such as one or more floating assets 620 orassets in the maritime facilities 622. In embodiments, a data handlinglayer 608 of the management platform 604 including data sources such asin the data storage layer 624 and from other inputs such as from themonitoring layer 614 that are collected with respect to any of the valuechain entities 652 including one more maritime assets. In embodiments,the data sources contain information used to populate a training setbased on a set of maritime activities of one or more of the maritimeassets and one of design outcomes, parameters, and data from one or moreof the data handling layers 608 is associated with the one or moremaritime assets. In embodiments, an artificial intelligence system suchas the adaptive intelligence layer 614 can be configured to learn on oneor more of the training sets obtained from the data sources from the oneor more data handling layers 608. In doing so, the artificialintelligence system can simulate one or more design attributes of one ormore of the maritime assets. The artificial intelligence system can alsogenerate one or more sets of design recommendations based on thetraining sets collected from the data sources. In embodiments, a digitaltwin system 1700 in the value chain network management platform 604 canprovides for visualization of one or more digital twins of one or moreof the maritime assets including detail generated by the artificialintelligence system of one or more of the design attributes incombination with the one or more sets of design recommendations.

In embodiments, the maritime assets can include one or more containerships. In embodiments, the maritime assets include one or more barges.In embodiments, the maritime assets include one or more components ofthe port infrastructure installed on or adjacent to land. Inembodiments, the maritime assets include one or more moored navigationunits deployed on water. In embodiments, the maritime assets include aship and the maritime activities include the forward speed of the shiprelative to water and weather conditions based on the parametersassociated with energy consumption of the propulsion units on the ship.

In embodiments, an information technology system includes a set ofintelligent systems for automatically populating a digital twin of amaritime value chain network entity based on data collected by the valuechain network management platform 604. In embodiments, the maritimevalue chain network entity is associated with one or more of thereal-world shipyards and the digital twin can be configured to representone or more of the real-world shipyards. In embodiments, the maritimevalue chain network entity is associated with a real-world maritime portand the digital twin can be configured to represent one or more of thereal-world maritime ports. In embodiments, the maritime value chainnetwork entity is associated with one or more of the container ships andthe digital twin can be configured to represent one or more of thecontainer ships. In embodiments, the maritime value chain network entityis associated with one or more of the barges and the digital twin can beconfigured to represent one or more of the barges.

In embodiments, the maritime value chain network entity is associatedwith one or more event investigations 7700 and the digital twin can beconfigured to at least partially represent the maritime value chainnetwork entity as it can act and interact with other assets during atimeline associated with one or more of the event investigations 7700.In embodiments, the maritime value chain network entity is associatedwith one or more legal proceedings 7702 and the digital twin can beconfigured to at least partially represent the maritime value chainnetwork entity as it can act and interact with other assets during atimeline associated with the one or more of the legal proceedings 7702.In embodiments, the data collected by a value chain network managementplatform relates to a casualty report 7704 and the digital twin of themaritime value chain network entity is configured to simulatepossibilities of a loss 7708 relevant to the casualty report 7704 basedon the data collected by a value chain network management platform.

In embodiments, the maritime value chain network entity is a portinfrastructure facility, wherein the data collected by a value chainnetwork management platform facilitates identifying theft or misuse ofthe port infrastructure facility by correlating data between a set ofdata collectors for one or more physical items 7710 in the portinfrastructure facility and the digital twin can be configured to detailthe one or more physical items 7710 of the port infrastructure facilityfor the at least one of the port infrastructure facility and the set ofoperators 7720.

In embodiments, the maritime value chain network entity is a containership that is moored to port infrastructure installed on or adjacent toland.

In embodiments, data collected by a value chain network managementplatform is based on at least a container ship having a forward speedrelative to water and weather conditions and parameters associated withenergy consumption of propulsion units on the container ship.

In embodiments, the value chain network management platform 604 includesan asset management application 814 associated with the value chainnetwork management platform and one or more maritime facilitiesconnected to a container ship.

In embodiments, the asset management application is associated with oneor more ships connected to barges.

In embodiments, the maritime value chain network entity is one or moreships and the digital twin can provide for visualization of a navigationcourse of one or more of the ships. In embodiments, the maritime valuechain network entity is one or more ships and the digital twin canprovide for visualization of an engine performance of one or more of theships. In embodiments, the maritime value chain network entity is one ormore ships and the digital twin can provide for visualization of a hullintegrity of one or more of the ships.

In embodiments, the digital twin can provide for visualization of aplurality of inspection points 7730 on the maritime value chain networkentity and maintenance histories 7732 associated with those inspectionpoints. In embodiments, the digital twin can further provide for thevisualization of the plurality of the inspection points 7730 on themaritime value chain network entity within geofenced parameters 7740 andmaintenance histories 7732 associated with those inspection points 7730.

In embodiments, the digital twin can further provide for details of aledger 7750 of activity associated with the visualization of theplurality of inspection points 7730 on the maritime value chain networkentity within geofenced parameters 7740 and maintenance historiesmardst832 associated with those inspection points 7730.

Control Tower and Enterprise Management Platform for Value Chain Network

In embodiments, the control tower may include or interface with anenterprise management platform (or “EMP”). In embodiments, an EMP may beconfigured to generate, integrate with, support, and/or or operate onone or more digital twins. In general, digital twins merge data frommultiple data sources into a model and representation of the salientcharacteristics of things, assets, systems, devices, machines,components, equipment, facilities, individuals or other entitiesmentioned throughout this disclosure or in the documents incorporatedherein by reference, such as, without limitation: machines and theircomponents (e.g., delivery vehicles, forklifts, conveyors, loadingmachines, cranes, lifts, haulers, trucks, loading machines, unloadingmachines, packing machines, picking machines, and many others, includingrobotic systems (e.g., physical robots, collaborative robots, “cobots”),drones, autonomous vehicles, software bots and many others); value chainprocesses, such as shipping processes, hauling processes, maritimeprocesses, inspection processes, hauling processes, loading/unloadingprocesses, packing/unpacking processes, configuration processes,assembly processes, installation processes, quality control processes,environmental control processes (e.g., temperature control, humiditycontrol, pressure control, vibration control, and others), bordercontrol processes, port-related processes, software processes (includingapplications, programs, services, and others), packing and loadingprocesses, financial processes (e.g., insurance processes, reportingprocesses, transactional processes, and many others), testing anddiagnostic processes, security processes, safety processes, reportingprocesses, asset tracking processes, and many others; wearable andportable devices, such as mobile phones, tablets, dedicated portabledevices for value chain applications and processes, data collectors(including mobile data collectors), sensor-based devices, watches,glasses, wearables, head-worn devices, clothing-integrated devices,bands, bracelets, neck-worn devices, AR/VR devices, headphones, and manyothers; workers, such as delivery workers, shipping workers, bargeworkers, port workers, dock workers, train workers, ship workers,distribution of fulfillment center workers, warehouse workers, vehicledrivers, business managers, engineers, floor managers, demand managers,marketing managers, inventory managers, supply chain managers, cargohandling workers, inspectors, delivery personnel, environmental controlmanagers, financial asset managers, process supervisors and workers (forany of the processes mentioned herein), security personnel, safetypersonnel and many others); suppliers, such as suppliers of goods andrelated services of all types, component suppliers, ingredientsuppliers, materials suppliers, manufacturers, and many others;customers, including consumers, licensees, businesses, enterprises,value added and other resellers, retailers, end users, distributors, andothers who may purchase, license, or otherwise use a category of goodsand/or related services; a wide range of operating facilities, such asloading and unloading docks, storage and warehousing facilities, vaults,distribution facilities and fulfillment centers, air travel facilities,including aircraft, airports, hangars, runways, refueling depots, andthe like, maritime facilities, such as port infrastructure facilities,such as docks, yards, cranes, roll-on/roll-off facilities, ramps,containers, container handling systems, waterways, locks, and manyothers), shipyard facilities, floating assets, such as ships, barges,boats and others), facilities and other items at points of origin and/orpoints of destination, hauling facilities, such as container ships,barges, and other floating assets, as well as land-based vehicles andother delivery systems used for conveying goods, such as trucks, trains,and the like; items or elements factoring in demand (i.e., demandfactors), including market factors, events, and many others; items orelements factoring in supply (i.e., supply factors), including marketfactors, weather, availability of components and materials, and manyothers; logistics factors, such as availability of travel routes,weather, fuel prices, regulatory factors, availability of space, such ason a vehicle, in a container, in a package, in a warehouse, in afulfillment center, on a shelf, or the like, and many others; retailers,including online retailers and others; pathways for conveyance, such aswaterways, roadways, air travel routes, railways and the like; roboticsystems, including mobile robots, cobots, robotic systems for assistinghuman workers, robotic delivery systems, and others; drones, includingfor package delivery, site mapping, monitoring or inspection, and thelike; autonomous vehicles, such as for package delivery; softwareplatforms, such as enterprise resource planning platforms, customerrelationship management platforms, sales and marketing platforms, assetmanagement platforms, Internet of Things platforms, supply chainmanagement platforms, platform-as-a-service platforms,infrastructure-as-a-service platforms, software-based data storageplatforms, analytic platforms, artificial intelligence platforms, andothers; and many others.

FIG. 68 is a schematic of an example environment of an enterprisemanagement platform 8000. In embodiments, the EMP 8000 may be integratedwith or accessible to a control tower via an application programminginterface (API). In some of these embodiments, the EMP 8000 may be aseries of microservices that are accessible to the control tower.

In embodiments, the EMP 8000 includes an enterprise configuration system8002, a digital twin system 8004, a collaboration suite 8006, an expertagent system 8008, and an intelligence service system 8010. Inembodiments, the EMP 8000 includes an API system 8014 that facilitatesthe transfer of data between one or more external systems and the EMP8000. In some embodiments, the intelligence service system 8010 includesan enterprise data store 8012 that stores data relating to enterprises,whereby the enterprise data is used by the digital twin system 8004, thecollaboration suite 8006, and/or the expert agent system 8008. Theenterprise data store 8012 may store any of a wide variety of data, suchas any data involved in the data pipeline described above and throughoutthis disclosure and the documents incorporated herein by reference. Inembodiments, the enterprise data store 8012 may store data that is beingused to update digital twins in real-time or substantially real time. Inembodiments, the enterprise data store 8012 may store databases, filesystems, folders, files, documents, transient data (e.g., real-time dataor substantially real-time data), sensor data, and the like.

In embodiments, the enterprise configuration system 8002 provides aninterface (e.g., a graphical user interface (GUI)) by which a user(e.g., an “on-boarding” user) may upload or otherwise provide datarelating to an enterprise. As used herein, an enterprise may refer to afor-profit or non-profit organization, company, governmental agency,non-governing organization, or the like. While described as anon-boarding user, the configuration of the enterprise managementplatform 8000 for a particular enterprise may be performed by any numberof users, including individuals associated with the enterprise,individuals associated with the EMP, and/or individuals associated witha third-party, such as a third host of a hosted EMP for an enterprise(which may be deployed on cloud resources, platform-as-a-service,software-as-a-service, multi-tenant data resources and/or similarresources) and/or a service provider.

In embodiments, the on-boarding user may define the types of enterprisedigital twins that may be generated by the digital twin system 8004 onbehalf of the enterprise being on-boarded. In embodiments, theon-boarding user may select different types of digital twins that willbe supported for the enterprise by the EMP 8000 via a GUI presented bythe enterprise configuration system 8002. For example, the user mayselect different types role-based digital twins from a menu of digitaltwin types, where the different types of role-based digital twinsinclude executive digital twins. As another example, the user may selecta type of organizational digital twin that is suitable for the user'sorganization, such as from a library of industry-specific ordomain-specific organizational templates. In some embodiments, each typeof executive digital twin has a predefined set of states (such term asreferenced herein encompassing states, entities, relationships,parameters, and other characteristics) that are depicted in therespective executive digital twin and predefined granularity levelsand/or other features for each state of the set. In some embodiments,the set of states that are depicted in the executive digital twin, thegranularity of each, and/or other features may be customized (e.g., bythe on-boarding user). In these embodiments, a user may define thedifferent states that are represented in each type of executive digitaltwin and/or the granularity for each of the states depicted in thedigital twin. For example, if the CEO of an enterprise has a financialbackground, the CEO may wish to have more financial data depicted in theCEO digital twin, such that the financial data is displayed at a highergranularity, or the CEO may wish to have access to underlyinginformation on financial models that are available to the digital twin,such as models used for determination of state information (e.g.,financial predictions or forecasts) or models used for augmentation ofstates (such as highlighting important deviations from expectations). Bycontrast, if the CEO has less financial experience or training, the CEOdigital twin may be configured with summary financial data and mayinclude prompts (which may be generated by an intelligent agent trainedon a set of enterprise and/or industry outcomes) to obtain CFO inputwhen states deviate from normal operating conditions. In this example,the CEO digital twin may be configured to depict the desired financialdata fields at a granularity level set defined by a user (e.g., thefinancial data may include various revenue streams, cost streams, andthe like). In another example, the CEO may have a technical background.In this example, the CEO digital twin may be configured to depict one ormore states related to the enterprise's product and R&D efforts, patentdevelopment, and product roadmaps at higher granularity levels. In yetanother example, a COO may be tasked with overseeing a product team, amarketing team, and an HR department of the enterprise. In this example,the COO may wish to view marketing-related states, productdevelopment-related states, and HR-related states at a lower granularitylevel. In this example, the COO digital twin may be configured to showvisual indicators that indicate whether any of the states are at acritical condition, an exceptional condition, or a satisfactorycondition. For instance, if employee turnover is very high and employeesatisfaction is low, the COO digital twin may depict that the HR-stateis at a critical level. In this configuration, the COO may select todrill down into the HR-state, where she may view the employee turnoverrate, hiring rate, and employee satisfaction survey results.

In another example, a COO or CTO digital twin may be configured torepresent and assist with discovery and management of interconnections,relationships and dependencies between enterprise operations andinformation technology. For example, a COO digital twin or a CFO digitaltwin may be configured to depict a set of operations entities andworkflows (e.g., flow diagrams that represent a production process, anassembly process, a logistics process, or the like), where entities(including human workers, robots, processing equipment, and otherassets) are depicted to operate on a set of inputs such as materials,components, products, containers and information) in order produce andhand off a set of outputs (of similar varied types) to the next set ofentities in the workflow for further processing. These may berepresented, for example, in a flow diagram that depicts each entity andits relationship in the flow to other entity. In embodiments, arole-based digital twin (such as a CIO digital twin) may also representan information technology system, such as representing sensors, IoTdevices, data collection and monitoring systems, data storage systems,edge and other computational systems, wired and wireless networkingsystems, and the like, including any of the types described throughoutthis disclosure. Each information technology component or system may bedepicted in the role-based digital twin, along with related data, suchas specifications, configuration parameters and settings, processingcapabilities, along with its relationship to other components, such asrepresenting data and networking connectivity to other components orsystems. In embodiments, a role-based digital twin may provide aconverged view that depicts operations technology entities andinformation technology entities in relation to each other, such asindicating which information technology entities are located with wiredor proximal wireless connectivity to which operational entities,indicating which informational technology entities are logicallyassociated to which operational entities (such as where cloud resources,computational resources, artificial intelligence resources, databaseresources, application resources, or other resources are provisioned tosupport or interact with operational entities, such as in virtualmachine, container or other logical relationships). In embodiments, theconverged view presented in the role-based digital twin may thus depictlocation-based and/or logical interconnections between operations andinformation technologies. In embodiments, alerts, such as indicatingfailure modes, congestion, delays, interruptions in service, poorlatency, diminished quality of service, bandwidth constraints, poorperformance on key performance indicators, downtime, or other issues maybe provided as augmentations or overlays of the converged informationtechnology and operations digital twin, so that the COO, CTO, CIO orother user may see interconnections between information technologyentities and operational entities that may be contributing to problems.Other types of issues that may be provided as augmentations or overlaysmay include alerts as to existing conditions and/or forecasts orpredictions of such conditions, such as by analytic systems orforecasting artificial intelligence systems, such as expert agentstrained to make such forecasts. In an example, if high latency in acontrol system for a warehouse is slowing down the process of pickingand packing goods due to a related edge computational node experiencingcongestion on an input data path, the user of the role-based digitaltwin may be alerted to the fact that operations are being adverselyimpacted by the congestion, and a recommendation may be presented toaugment, update, upgrade, or replace either the system providingconnectivity to the edge node or the edge node itself. Thus, a convergeddigital twin of operations technology entities and informationtechnology entities may provide for insight into how an executive mayadjust operations and/or information technology to improve resultsand/or avoid anticipated problems before they become catastrophicfailures.

In embodiments, a user (e.g., an on-boarding user) may connect one ormore data sources 8020 to the EMP 8000. Examples of data sources 8020that may be connected to the EMP may include, but are not limited to, asensor system 8022 (e.g., a set of IoT sensors), a sales database 8024that is updated with sales figures in real time, a customer relationshipmanagement (CRM) system 8026, a content marketing platform 8028, newswebsites 8048, a financial database 8030 that tracks costs of thebusiness, surveys 8032 (e.g., customer satisfaction and/or employeesatisfaction surveys), an org chart 8034, a workflow management system8036, customer databases 1S40 that store customer data, external datafeeds (such as news feeds, public relations feeds, weather feeds, tradedata, pricing data, market data, and the like), data obtained byspidering, webscraping, or otherwise parsing website and social mediasites, data obtained by crowdsourcing, and/or data from many and variousthird-party data sources 8038 that store third-party data. The datasources 8020 may include additional or alternative data sources withoutdeparting from the scope of the disclosure. Once the user has definedthe configuration of each respective executive digital twin, where theconfiguration includes the selected states to be depicted (which mayinclude entities, relationships, and characteristics), the features thatare to be enabled, and/or the desired granularity of each state, theuser may then define the data sources 8020 that are fed into therespective executive digital twin, including any of the data sources inthe data pipeline described above. In some embodiments, data from one ormore of the data sources may be fused and/or analyzed before being fedinto a respective digital twin.

In some embodiments, the on-boarding user may select among various typesof enterprise digital twins that are supported for the enterprise,including environment digital twins, information technology digitaltwins, operations digital twins, organizational digital twins, supplychain digital twins, product digital twins, facility digital twins,customer digital twins, cohort digital twins and/or process digitaltwins, among others. In some of these embodiments, the user may definethe data sources used to generate these digital twins and to update theenterprise digital twins. In embodiments, the user may define anyphysical locations that will be represented as an environment digitaltwin (which may be a digital twin of a facility or other suitableenvironments). For example, the user may define manufacturing facilities(e.g., factories), shipping facilities, warehouses, office buildings,and the like. Each facility may be given a location (which may include alogical and/or virtual location and/or a geo-location) and anidentifier, such as a name and type description. In embodiments, theenterprise configuration system 8002 may assign an identifier to eachfacility and may associate the location of the facility with theidentifier. In embodiments, the user may define the types of objectsthat are included in the environment and/or may be found within anenvironment. For example, the user may define the types of enterpriseresources (e.g., factory, warehouse, or distribution center equipmentand machines, assembly lines, conveyors, vehicles, robots, high-lows,and the like, IT systems, workers, and many others) that are in theenvironment, the types of products, materials and components that aremade in, stored in, moved around, assembled, used as inputs within,produced in, sold from, and/or received in the environment, the types ofsensors/sensor kits and/or data collection, storage and/or processingdevices that are used in the environment, the workers and workflowsinvolved, and the like. Examples of how environment and process digitaltwins are generated and updated may be found in the U.S. ProvisionalApplication No. 62/931,193, filed Nov. 5, 2019, entitled Methods andSystems of Value Chain Network Management Platform and U.S. ProvisionalApplication No. 62/969,153, filed Feb. 3, 2020, entitled Methods andSystems of Value Chain Network Management Platform, the contents ofwhich are herein incorporated by reference.

In embodiments, the enterprise configuration system 8002 (in combinationwith the digital twin system 8004) is configured to generateorganizational digital twins that represent an organizational structureof an enterprise. In some embodiments, the organizational digital twinmay depict individuals/roles occupying the management and expert levelsof an enterprise. Alternatively, the organizational digital twin mayinclude a workforce digital twin that represents the entire workforce ofan enterprise, including all the employees and/or contractors of theenterprise, or a defined part thereof. For example, in an enterprisesetting, workforces may include a logistics workforce, a warehouseworkforce, a distribution workforce, a reverse logistics workforce, adelivery workforce, a factory operations workforce, a plant operationsworkforce, a resource extraction operations workforce, a networkoperations workforce (e.g., for operating internal networks of anindustrial enterprise), a sales workforce, a marketing workforce, anadvertising workforce, a retail workforce, an R&D workforce, atechnology workforce, an engineering workforce, and/or the like. Inanother example, with respect to a value chain network, workforces mayinclude a supply chain management workforce, a logistics planningworkforce, a vendor management workforce, and the like. In anotherexample, in the context of a marketplace setting, workforces may includea brokering workforce for a marketplace, a trading workforce for amarketplace, a trade reconciliation workforce for a marketplace, atransactional execution workforce for a marketplace, and/or the like.Enterprises may include additional or alternative workforces. In someembodiments, an organizational digital twin may include management-levelroles within a workforce. Examples of management-level roles of anenterprise include a CEO role, a COO role, a CFO role, a counsel role, aboard member role, a CTO role, an information technology manager role, achief information officer role, a chief data officer role, an investorrole, an engineering manager role, a project manager role, an operationsmanager role, a business development role. Furthermore, themanagement-level roles of a workforce may include a factory managerrole, a factory operations role, a factory worker role, a power plantmanager role, a power plant operations role, a power plant worker role,an equipment service role, and an equipment maintenance operator role.In a value chain context, the management-level roles of a workforce mayinclude a chief marketing officer role, a product development role, asupply chain manager role, a customer role, a supplier role, a vendorrole, a demand management role, a marketing manager role, a salesmanager role, a service manager role, a demand forecasting role, aretail manager role, a warehouse manager role, a salesperson role, and adistribution center manager role. In the context of marketplaces, themanagement-level roles of a workforce may include a market maker role,an exchange manager role, a broker-dealer role, a trading role, areconciliation role, a contract counterparty role, an exchange ratesetting role, a market orchestration role, a market configuration role,and a contract configuration role. It is appreciated that not all of theroles defined above apply to a particular workforce type. Furthermore,some roles may be associated with different types of workforces.

In some embodiments, an organizational digital twin may furtherincorporate data access rules for different divisions and/or roleswithin the organization. For example, the CEO may be granted access tomost or all of the organization's data, the CFO may be granted access tofinancial-related data and restricted from viewing R&D data, the CTO maybe granted access to R&D-related data and restricted from viewingfinancial data, members of the engineering team may be restricted inaccessing financial related data, or the like. Similar rules may beapplied to access to features, such as analytic models, artificialintelligence systems, intelligent agents, and the like, includingrole-based or identity-based control of the ability to view results, toconfigure inputs, to configure or adjust models (e.g., weights, inputs,or processing functions), to undertake control actions, or the like. Insome embodiments, the EMP may utilize the organizational digital twinwhen determining the level of access a particular individual may begranted and/or whether to deny certain types of access to theindividual. In some embodiments, the access rights may limit the typesof data that particular users can access, such as information about eachindividual listed in the organizational digital twin (e.g., salary,start date, availability, work status, and the like). For example, lowerlevel employees may not be granted access to sensitive information, suchas financial data, product strategies, marketing strategies, tradesecrets, or the like. In some embodiments, certain users may be grantedpermission to change the access rights of other employees, which may bereflected in the organizational digital twin. For example, certainexecutives and managers may be granted permission to grant access rightsto members of their respective teams when working on certain projects.

In embodiments, the enterprise configuration system 8002 receives anorganization chart (“org chart”) definition of an enterprise andgenerates an organizational digital twin based on the org chartdefinition. In embodiments, the org chart definition may define thebusiness units/departments of the enterprise, the reporting structure ofthe enterprise, various roles of the enterprise/within each businessunit, and the individuals in the respective roles. In some embodiments,the user can upload the enterprise's org chart to the EMP 8000 via theenterprise configuration system 8002. Additionally or alternatively, theuser can define the structure of the org chart (e.g., roles, businessunits, reporting structure) and may populate the various roles withnames and/or other identifiers of the individuals filling the respectiveroles defined in the org chart. In some embodiments, the enterpriseconfiguration system 8002 may access an enterprise resource planningsystem 8044 and/or an HR system 8046 of the enterprise to obtainorganizational data of the enterprise, such as the roles of theenterprise, the individuals that fill the roles, the salaries of theindividuals that fill the roles, the reporting structure of theenterprise, and the like. In these embodiments, the digital twin system8004 (discussed below) may continue to communicate with the ERP system8044 and/or HR system 8046 to receive the data needed to maintain theorganizational digital twin in a real-time or near-real-time manner.

In embodiments, the enterprise configuration system 8002 (in cooperationwith the digital twin system 8004, discussed below) may generate anorganizational digital twin of the enterprise based on the org chartdefinition and the individuals that populate the roles within the orgchart definition. In embodiments, a user may define one or morerestrictions, permissions, and/or access rights of the individualsindicated in the organizational digital twin via the enterpriseconfiguration system 8002. In embodiments, a restriction may define oneor more types of data or features that a particular user or group ofusers is not allowed to access (either directly or in a digital twin).In embodiments, an access right may define one or more types of data orfeatures that a particular user or group of users may access and thetype of access that a user or group of users can access. In embodiments,a permission may define operations that a user or a group of users mayperform with respect to the EMP 8000. In embodiments, one or more of theaccess rights, permissions, and restrictions may be definedgeographically and/or temporally limited. For example, some types ofdata or features may only be viewed or otherwise accessed in certainareas (e.g., sensitive data may only be viewed in the corporate offices)or at certain times (e.g., during Board meetings). In embodiments, therestrictions, permissions, and/or access rights may be set with respectto roles or the users themselves. As such, defining access rights,permissions, and/or restrictions for a user or a group of users may alsoinclude defining access rights, permissions, and/or restrictions to arole and/or business unit within the enterprise. In embodiments, theorganizational digital twin may be deployed to manage the rights,permissions, and/or restrictions for the users of an enterprise.Furthermore, in embodiments, the organizational digital twin may definethe types of role-based digital twins (and other enterprise digitaltwins) that various users may have access to. In some embodiments, theorganizational digital twin may depict additional or alternativeinformation.

In embodiments, the digital twin system 8004 is configured to generate,update, and serve enterprise digital twins of an enterprise. In someembodiments, the digital twin system 8004 is configured to generate andserve role-based digital twins on behalf of an enterprise and may servethe role-based digital twins to a client device 8050 (e.g., a mobiledevice, a tablet, a personal computer, a laptop, AR/VR-enabled device,workflow-specific device or equipment, or the like). As discussed,during the configuration phase, a user may define the different types ofdata and the corresponding data sources, data sets, and features thatare used to generate and maintain each respective type of the differenttypes of enterprise digital twins. Initially, the digital twin system8004 configures the data structures that support each type of enterprisedigital twin, including any underlying data sources/databases (e.g., SQLdatabases, graph databases, relational databases, distributed databases,blockchains, distributed ledgers, data feeds, data streams, and thelike) that store or produce data that is ingested by the respectiveenterprise digital twins. Once the data structures that support adigital twin are configured, the digital twin system 8004 receives datafrom one or more data sources 8020. In embodiments, the digital twinsystem 8004 may structure and/or store the received data in one or moredatabases. When a specific digital twin is requested (e.g., by a uservia a client application 8052 or by a software component of the EMP8000), the digital twin system may determine the views that arerepresented in the requested digital twin and may generate the requesteddigital twin based on data from the configured databases and/orreal-time data received via an API. The digital twin system 8004 mayserve the requested digital twin to the requestor (e.g., the clientapplication or a backend software component of the EMP 8000). After anenterprise digital twin is served, some enterprise digital twins may besubsequently updated with real-time data received via the API system8014. In embodiments, an API may provide information to the datapipeline as to the type of data required for the digital twin, such thatthe data pipeline may be configured (by a user, or by anautomated/intelligence systems) to handle the data effectively. Forexample, the data pipeline may be configured to deliver data over a datapath that uses an appropriate protocol for efficient delivery,delivering the data over a cost-appropriate path (e.g., an inexpensivepath for data that does not require low latency or real-time updating),or the like. Thus, in some embodiments, configuration of a digital twinmay include providing inputs as to the requirements of the digital twinfor low-latency, high quality-of-service, high accuracy, highgranularity, high reliability, or the like, based on, for example, thepriority of the mission served by the data type. In embodiments, anintelligent expert agent (or “intelligent agent” or “expert agent”) maybe trained on a training set of configurations of inputs to one or moredata pipelines that were previously configured by experts, such that theintelligent agent may learn to automatically configure APIs for digitaltwins to provide appropriate inputs to data pipelines for subsequentdigital twins involving similar or analogous workflows for similar oranalogous roles, identities, industries and/or domains. In embodiments,such training of an intelligent agent may include learning as tospecific user interactions, such as learning which users within a roleuse which types of data at what times and for what purposes, such thatdata resources are appropriately allocated to support actual userrequirements. For example, an automated intelligent agent managing theconfiguration of a data pipeline for a COO digital twin may learn thatan operations executive (e.g., a COO user) checks production data foreach facility at the end of each eight-hour shift (e.g., after 5:00 pm),such that mid-shift data updates are delivered over lower-cost dataresources, but end-of-shift data is delivered over low-latency datapaths that have high reliability and quality-of-service. Continuing thisexample, the intelligent agent may determine the frequency at which theproduction data is updated with respect to the COO digital twin, suchthat the COO digital twin is updated less frequently in the mornings andmid-afternoons, but is updated more frequently at the end of businesshours. In embodiments, the intelligent agent may be configured withbusiness logic that defines overall strategies (e.g., when to uselow-latency networks v. higher-latency networks and/or how often toupdate a certain type of data within a particular digital twin) andcustomized based on the preferences and use by the end user of thedigital twin, whereby the overall strategies may be learned fromtraining data sets obtained from experts and/or may be hard-coded by adeveloper, and the customization piece may be learned from monitoringthe use of the digital twin by the end intended user (e.g., when shetypically checks the production data of each facility). Additional oralternative examples of such data prioritization strategies and/or otherconfiguration strategies should be understood to be encompassed herein.For example, upon receipt of inputs as to performance requirements,artificial intelligence capabilities of the data pipeline that isintegrated with, linked to, or supporting of the EMP 100 mayautomatically or under user control employ techniques to provideappropriate resources at the right time and place, including, but notlimited to: adaptive coding of data path transmissions between networkeddata communication nodes; adaptive filtering, repeating andamplification of RF/wireless signals (including software-implementedbandpass filtering); dynamic allocation of use of cellular and otherwireless spectrum, adaptive, ad-hoc, cognitive management of wirelessmesh network nodes; adaptive data storage; cost-based routing ofwireless and wired signals; priority-based routing; channel- andperformance-aware protocol selection for communications; context-awareallocation of computational resources, serverless computational systems,adaptive edge computational systems, channel-aware error correction,smart-contract-implemented network resource allocation; and/or othersuitable techniques.

In embodiments, the digital twin system 8004 may be further configuredto perform simulations and modeling with respect to the enterprisedigital twins. In embodiments, the digital twin system 8004 isconfigured to run data simulations and/or environment simulations usinga digital twin. For example, a user may, via a client device, instructthe digital twin system 8004 to perform a simulation with respect to oneor more states and/or workflows depicted in a digital twin. The digitaltwin system 8004 may run the simulation on the digital twin and maydepict the results of the simulation in the digital twin. In thisexample, the digital twin may need to simulate at least some of the dataused to run the simulation of the environment, so that there is reliabledata when performing the requested environment simulation. The digitaltwin system 8004 is discussed in greater detail throughout thedisclosure.

In embodiments, the collaboration suite 8006 provides a set of variouscollaboration tools that may be leveraged by various users of anenterprise. The collaboration tools may include video conferencingtools, “in-twin” collaboration tools, whiteboard tools, presentationtools, word processing tools, spreadsheet tools, and the like. Inembodiments, an “in-twin” collaboration tool allows multiple users toview and collaborate within a digital twin. For example, in embodiments,the collaboration tools may include an in-twin collaboration tool thatthat enables a digital twin experience and a collaboration experiencewithin the same interface (e.g., within a AR/VR-enabled user interface,a standard GUI, or the like), such as where collaboration entities andevents (such as version-controlled objects, comment streams, editingevents and other changes) are represented within the digital twininterface and linked to digital twin entities. For example, multipleusers may be granted access to view an environment digital twin of afacility, such as a warehouse or factory, via an in-twin collaborationtool. Once viewing the environment digital twin, the users may thenchange one or more features of the environment depicted in theenvironment digital twin and may instruct the digital twin system toperform a simulation. In this example, the results of the simulation maybe presented to the users in the digital twin and may be automaticallypopulated into a shared document (e.g., a spreadsheet or presentationdocument). Users may collaborate in additional manners with respect to adigital twin, as will be discussed throughout the disclosure. Forexample, in some embodiments, the collaboration suite 8006 may allow auser to call a video conference with another user, where the users seeeach other and see aspects of a specific digital twin that relates tothe topics of discussion for the conference. In this example, users may,for example, see a representation of workpiece under discussion and seeeach other, so that a user can see gestures or indications from anotheruser about how the workpiece should be acted upon. In another example, aconferencing feature of the twin may show participants in a view of aset of environments of facilities by their locations, so that users canrecognize which participants may have closest proximity to relevantassets that are the subject of collaboration. In some embodiments, thecollaboration suite 8006 interfaces with third-party applications,whereby data may be imported to and/or from the third-party application.For example, in collaborating on a Board presentation, differentexecutives may export data from their respective executive digital twininto a shared presentation file (e.g., PowerPoint™ file or Google™ slidepresentation). In another example, a first user (e.g., the CEO of anenterprise) may request certain information (e.g., financial projectionsfor the enterprise) from a second user (e.g., the CTO of the enterprise)via a first executive digital twin configured for the first user (e.g.,a CEO digital twin of the enterprise). In response, the second user mayupload/export the requested data from a second executive digital twinthat was configured for the second user (e.g., the CTO) to the EMP 8000(e.g., to the collaboration suite 8006 and/or the digital twin system8004, which may then update the executive digital twin configured forthe first user. Additional examples and descriptions of thecollaboration suite 8006 and underlying collaboration tools arediscussed throughout the disclosure.

In embodiments, the collaboration suite 8006 may be configured tointerface with the digital twin system 8004 (e.g., independent of orunder control of the digital twin system 8004) to provide role-specificviews and other features within a collaboration environment and/orworkflow of a collaboration tool, such that different participants inthe same collaboration environment and/or workflow experience differentviews or features of the same digital twin entities and/or workflows.For example, a CFO may collaborate with a COO and a CTO about thepossible replacement of an internal system or a piece of machinery orequipment, where the current system, machinery or equipment and/or thepotential replacement system, machinery, or equipment is/are representedin the digital twin by visual and other elements. During collaboration,the collaboration suite 8006 may recognize the identities/roles of theCFO, COO and CTO and may automatically configure their respectivecollaboration views into the example digital twin based on those roles.For example, the CFO may be presented with a view that is augmented withfinancial data, such as the cost of the item and various possiblereplacements, terms and conditions of leasing agreements, depreciationinformation, information on the financial impacts on productivity, orthe like. Meanwhile, the collaboration suite 8006 may present the COOwith information depicting the relationship of the item to operationalprocesses, such as linkages to other systems involved in a productionline, timing information (such as scheduled downtimes for a facility)and the like. In this example, the CTO may be presented with performancespecifications and capability information for an item and variouspossible replacements, including, for example, compatibility informationthat indicates the extent to which various possible replacements arecompatible with other items represented in the digital twin (includingphysical/mechanical compatibility, data compatibility, softwarecompatibility, and many other forms of technology compatibility),reviews and ratings, and other technical information. Each executiveuser may be presented with respective information that is in therespective user's “native language” (e.g., information that is tailoredto each executive's respective expertise and needs) and with respectiveviews and/or features that are comfortable for that user, while thegroup can collaborate (in live or asynchronous modes) to raise issues,engage in commentary and dialog, perform analysis (including simulationsas described herein) to arrive at a decision (e.g., about selection andtiming of a replacement, or an alternative like a repair) that isfinancially prudent, operationally effective, and technologically sound.Thus, a role-sensitive collaboration environment integrated with respectto a shared enterprise digital twin enables collaboration around digitaltwin entities and workflows while allowing users to engage withrole-sensitive views and features. In embodiments, the collaborationsuite 8006 and or other systems of the EMP 8000 (e.g., the digital twinsystem 8004) may access a semantic model of an enterprise taxonomy toautomatically generate and/or provide information that is presented in ashared digital twin (such as role-specific augmentation of entities withtext or symbols that is derived from data or metadata based on stateinformation or other data). In embodiments, the enterprise taxonomy maybe learned by the EMP 8000 via an analysis of data provided by theenterprise or may be manually uploaded by a user (e.g., a configuratinguser associated with the enterprise). The information in the digitaltwin may be presented with a role-specific understanding of thetaxonomy, such as where the same entity (e.g., a piece of equipment) isgiven a different name by different groups in the enterprise (e.g.,referred to as an “asset” by the finance department and a “machine” bythe operations team) and/or where attributes of the entity or relatedworkflows use different terminology, codes, symbols, or the like thatare role-specific or group-specific. In embodiments, the collaborationsuite 8006 may automatically enable translation of terminology betweenroles, such as translating commentary that uses the name of an entity orthat describes attributes of the entity from one role-specific form toanother role-specific form. Automatic translation may presentalternative terms together (e.g., as the “asset/machine” or “codered/urgent”). In embodiments, automated translation may be performed bytranslation models (e.g., enterprise-specific translation models) thatare trained by machine learning or similar techniques, whereby thetranslation models may be leveraged to provide automated translation forrole-sensitive entity, workflow and attribute presentation. Inembodiments, the translation models may be trained using a training dataset of translations generated by human experts and/or by unsupervisedlearning techniques that operate on the data of the enterprise toidentify associations between different terms used by different rolesand/or groups to describe the same thing. In embodiments, translationmodels may be seeded by an explicit translation model or may beaccomplished by deep learning or similar techniques known to those ofskill in the art.

In embodiments, the expert agent system 8008 trains expert agents thatperform/recommend actions on behalf of an expert. An expert agent may bea software module that implements and/or leverages artificialintelligence services to perform/recommend actions on behalf of or inlieu of an expert. In embodiments, an expert agent may include one ormore machine-learned models (e.g., neural networks, prediction models,classification models, Bayesian models, Gaussian models, decision trees,random forests, and the like, including any of the artificialintelligence systems, expert systems, or the like described throughoutthis disclosure and/or the documents incorporated herein by reference)that perform machine-learning tasks, including robotic processautomation, in connection with a defined role. Additionally oralternatively, an expert agent may be configured with artificialintelligence rules that determine actions in connection with a definedrole. The artificial intelligence rules may be programmed by a user ormay be generated by the expert agent system 8008. An expert agent may beexecuted at a client device 8050 and/or may be executed by or by asystem that is linked to or integrated with the EMP 8000. Inembodiments, the expert agent may be accessed as a service (e.g., via anAPI), such as in a service-oriented architecture, which in embodimentsmay be integrated with the EMP as service that is part of amicroservices architecture. In embodiments, where an expert agent is atleast partially executed at a client device, the EMP 8000 may train anexecutive agent and may serve the trained executive agent to a clientapplication 8052. In embodiments, an expert agent may be implemented asa container (e.g., a Docker container), virtual machine, virtualizedapplication, or the like that may execute at the client device 8050 orat the EMP 8000. In embodiments, the expert agent is further configuredto collect and report data to the expert agent system 8008, which theexpert agent system 8008 uses to train/reinforce/reconfigure the expertagent. Many examples of such training are described throughout thisdisclosure and many others are intended to be encompassed by thedisclosure.

In some embodiments, the expert agent system 8008 (working in connectionwith the artificial intelligence services system 8010) may train expertagents (e.g., executive agents and other expert agents), such as usingrobotic process automation techniques, machine learning techniques, orother artificial intelligence or expert systems as described throughoutthis disclosure and/or the documents incorporated by reference herein toperform one or more executive actions on behalf of respective users,such as executives or other users who are responsible for undertakingactivities that are automated by the robotic process automation or othertechniques. In some of these embodiments, a client application 8052 mayexecute on a client device 8050 (e.g., a user device, such as a tablet,an AR and/or VR headset, a mobile device, or a laptop, an embeddeddevice, an enterprise server, or the like) associated with a user (e.g.,an executive, an administrative assistant of the executive, a boardmember, a role-based expert, a manager, a worker, or any other suitableemployee or affiliate). In embodiments, the client application 8052 mayrecord the interactions of a user with the client application 8052 andmay report the interactions to the expert agent system 8008. In theseembodiments, the client application 8052 may further record and reportfeatures relating to the interaction, such as any stimuli or inputs thatwere presented to the user, what the user was viewing at the time of theinteraction, the type of interaction, the role of the user, whether theinteraction was requested by someone else, the role of the individualthat requested the interaction, contextual information, stateinformation, workflow information, event information, and the like. Theexpert agent system 8008 may receive the interaction data and relatedfeatures and may generate, train, configure, and/or update an executiveagent based thereon. In embodiments, the interactions may beinteractions by the user with an enterprise digital twin (e.g., anenvironment digital twin, a role-based digital twin, a process digitaltwin, and the like). In embodiments, the interactions may beinteractions by the user with data, such as sensor data (e.g., vibrationdata, temperature data, pressure data, humidity data, radiation data,electromagnetic radiation data, motion data, and/or the like) and/ordata streams collected form physical entities of the enterprise (e.g.,machinery, a building, a shipping container, or the like), data fromvarious enterprise and/or third-party data sources (as describedthroughout this disclosure and incorporated documents), entity data(such as characteristics, features, parameters, settings,configurations, attributes and the like), workflow data (such as timing,decision steps, events, tasks activities, dependencies, resources, orthe like), and many other types of data. For example, a user may bepresented with sensor data from a particular piece of machinery orequipment and, in response, may determine that a corrective action to betaken with respect to the piece of machinery or equipment. In thisexample, the expert agent may be trained on the conditions that causethe user to take a corrective action as well as instances where the userdid not take corrective actions. In this example, the expert agent maylearn the circumstances in which corrective action is taken.

In embodiments, the expert agent system 8008 may train expert agentsbased on user interactions with network entities and/or computationentities. For example, the expert agent system 8008 may train an expertagent to learn the manner by which an IT expert diagnoses and handles asecurity breach. In this example, the expert agent may be trained tolearn the steps undertaken by the expert to diagnose a security breach,the individuals within the enterprise that the security breach isreported to, and any actions undertaken by the expert to resolve thesecurity breach.

In embodiments, the types of actions that an expert agent may be trainedto perform/recommend include: selection of a tool, selection of a task,selection of a dimension, setting of a parameter, configuration ofsettings, flagging an item for review, providing an alert, providing asummary report of data, selection of an object, selection of a workflow,triggering of a workflow, ordering of a process, ordering of a workflow,cessation of a workflow, selection of a data set, selection of a designchoice, creation of a set of design choices, identification of a failuremode, identification of a fault, identification of an operating mode,identification of a problem, selection of a human resource, selection ofa workforce resource, providing an instruction to a human resource, andproviding an instruction to a workforce resource, amongst other possibletypes of actions. In embodiments, an expert agent may be trained toperform other types of tasks, such as: determining an architecture for asystem, reporting on a status, reporting on an event, reporting on acontext, reporting on a condition, determining a model, configuring amodel, populating a model, designing a system, designing a process,designing an apparatus, engineering a system, engineering a device,engineering a process, engineering a product, maintaining a system,maintaining a device, maintaining a process, maintaining a network,maintaining a computational resource, maintaining equipment, maintaininghardware, repairing a system, repairing a device, repairing a process,repairing a network, repairing a computational resource, repairingequipment, repairing hardware, assembling a system, assembling a device,assembling a process, assembling a network, assembling a computationalresource, assembling equipment, assembling hardware, setting a price,physically securing a system, physically securing a device, physicallysecuring a process, physically securing a network, physically securing acomputational resource, physically securing equipment, physicallysecuring hardware, cyber-securing a system, cyber-securing a device,cyber-securing a process, cyber-securing a network, cyber-securing acomputational resource, cyber-securing equipment, cyber-securinghardware, detecting a threat, detecting a fault, tuning a system, tuninga device, tuning a process, tuning a network, tuning a computationalresource, tuning equipment, tuning hardware, optimizing a system,optimizing a device, optimizing a process, optimizing a network,optimizing a computational resource, optimizing equipment, optimizinghardware, monitoring a system, monitoring a device, monitoring aprocess, monitoring a network, monitoring a computational resource,monitoring equipment, monitoring hardware, configuring a system,configuring a device, configuring a process, configuring a network,configuring a computational resource, configuring equipment, andconfiguring hardware. As discussed, an expert agent is configured todetermine an action and may output the action to a client application8052. Examples of an output of an expert agent may include arecommendation, a classification, a prediction, a control instruction,an input selection, a protocol selection, a communication, an alert, atarget selection for a communication, a data storage selection, acomputational selection, a configuration, an event detection, aforecast, and the like. Furthermore, in some embodiments, the expertagent system 8008 may train expert agents to provide training and/orguidance rather in addition to or in lieu of outputting an action. Inthese embodiments, the training and/or guidance may be specific for aparticular individual or role or may be used for other individuals.

In embodiments, the expert agent system 8008 is configured to providebenefits to experts that participate in the training of expert agents.In some embodiments, the benefit is a reward that is provided based onthe outcomes stemming from the user of an expert agent that is trainedat least in part based on actions by the expert user. In someembodiments, the benefit is a reward that is provided based on theproductivity of the expert agent. For example, if an expert agenttrained by an individual is leveraged in connection with a set of usersin the enterprise (or outside the enterprise), an account with theindividual may be credited with a benefit such as a cash rewards, stockrewards, gift card rewards, or the like. As the expert agent is usedmore, the benefit to the individual may be increased. In someembodiments, the benefit is a reward that is provided based on a measureof expertise of the expert agent. For example, individuals having a moresought after/valuable skill may be awarded greater benefits thanindividuals having a less sought after/valuable skill. In someembodiments, the benefit is a share of the revenue or profit generatedby, or cost savings resulting from, the work produced by the expertagent. In some embodiments, the benefit is tracked using a distributedledger (e.g., a blockchain) that captures information associated with aset of actions and events involving the expert agent. In some of theseembodiments, a smart contract may govern the administration of thereward to the expert user.

In some embodiments, a set of expert agents trained by the expert agentsystem 8008 may be deployed as a double of at least a portion of aworkforce of an enterprise, where the expert agents perform tasks ofdifferent roles within the enterprise. In some of these embodiments, theexpert agents may be trained upon a training set of data that includes aset of interactions by members of a defined workforce of the enterpriseduring performance of the defined set of roles of the defined workforce(e.g., interactions with physical entities, digital twins, sensor data,data streams, computational entities, and/or network entities, amongmany others). In some embodiments, the interactions may be parsed toidentify a chain of operations performed by the workforce and/or a chainof reasoning, whereby the chain of operations and/or chain of reasoningare used to train the expert agents. In some embodiments, theinteractions may be parsed to identify types of processing performed bythe workforce upon a set of information, whereby the type of processingis embodied in the configuration of the respective expert agents.Examples of workforces may include, factory operations, plantoperations, resource extraction operations, network operations (e.g.,responsible for operating a network for an industrial enterprise), asupply chain workforce, a logistics planning workforce, a vendormanagement workforce, a brokering workforce for a marketplace, a tradingworkforce for a marketplace, a trade reconciliation workforce for amarketplace, a transactional execution workforce for a marketplace, andthe like.

In some embodiments, the expert agent system 8008 and/or a clientapplication 8052 can monitor outcomes related to the user's interactionsand may reinforce the training of the expert agent based on theoutcomes. For example, each time the user takes a corrective action, theexpert agent system 8008 may determine the outcome (e.g., whether aparticular condition or issue was resolved) and whether the outcome is apositive outcome or a negative outcome. The expert agent system 8008 maythen retrain the expert agent based on the outcome. Examples of outcomesmay include data relating to at least one of a financial outcome, anoperational outcome, a fault outcome, a success outcome, a performanceindicator outcome, an output outcome, a consumption outcome, an energyutilization outcome, a resource utilization outcome, a cost outcome, aprofit outcome, a revenue outcome, a sales outcome, and a productionoutcome. In these embodiments, the expert agent system 8008 may monitordata obtained from the various data sources after an action is taken todetermine an outcome (e.g., sales increased/decreased and by how much,energy utilization decreased/increased and by how much, costsdecreased/increased and by how much, revenue increased/decreased and byhow much, whether consumption decreased/increased and by how much,whether a fault condition was resolved, and the like). The expert agentsystem 8008 may include the outcome in the training data set associatedwith the action undertaken by the expert that resulted in the outcome.

In some embodiments, the expert agent system 8008 receives feedback fromusers regarding respective executive agents. For example, in someembodiments, a client application 8052 that leverages an expert agentmay provide an interface by which a user can provide feedback regardingan action output by an expert agent. In embodiments, the user providesthe feedback that identifies and characterizes any errors by the expertagent. In some of these embodiments, a report may be generated (e.g., bythe client application or the EMP 8000) that indicates the set of errorsencountered by the expert. The report may be used to reconfigure/retrainthe executive agent. In embodiments, the reconfiguring/retraining anexecutive agent may include removing an input that is the source of theerror, reconfiguring a set of nodes of the artificial intelligencesystem, reconfiguring a set of weights of the artificial intelligencesystem, reconfiguring a set of outputs of the artificial intelligencesystem, reconfiguring a processing flow within the artificialintelligence system, and/or augmenting the set of inputs to theartificial intelligence system.

In embodiments, the expert agent may be configured to, at leastpartially, operate as a double of the expert for a defined role withinan enterprise. In these embodiments, the expert agent system 8008 trainsan expert agent based on a training data set that includes a set ofinteractions by a specific expert worker during the performance of theirrespective role. For example, the set of interactions that may be usedto train the executive agent may include interactions of the expert withthe physical entities of an enterprise, interactions of the expert withan enterprise digital twin, interactions of the expert with sensor dataobtained from a sensor system of the enterprise, interactions of theexpert with data streams generated by the physical entities of theenterprise, interactions of the expert with the computational entitiesof the enterprise, interactions of the expert with the network entities,and the like. In some embodiments, the expert agent system 8008 parsesthe training data set of interactions to identify a chain of reasoningof the expert upon a set of interactions. In some of these embodiments,the chain of reasoning may be parsed to identify a type of reasoning ofthe worker, which may be used as a basis for configuring/training theexpert agent. For example, the chain of reasoning may be a deductivechain of reasoning, an inductive chain of reasoning, a predictive chainof reasoning, a classification chain of reasoning, an iterative chain ofreasoning, a trial-and-error chain of reasoning, a Bayesian chain ofreasoning, a scientific method chain of reasoning, and the like. In someembodiments, the expert agent system parses the training data set ofinteractions to identify a type of processing undertaking by the expertin analyzing the set of interactions. For example, types of processingmay include audio processing in analyzing audible information, tactileor “touch” processing in analyzing physical sensor information,olfactory processing in analyzing chemical sensing information, textualinformation processing in analyzing text, motion processing in analyzingmotion information, taste processing in analyzing chemical information,mathematical processing in mathematically operating on numerical data,executive manager processing in making executive decisions, creativeprocessing when deriving alternative options, analytic processing whenselecting from a set of options, and the like.

In embodiments, the expert agents include executive agents that aretrained to output actions on behalf of executive and/or an administratorof an executive. In these embodiments, an expert agent may be trainedfor executive roles, such that a user in an executive role can train theexecutive agent by performing their respective role. For example, anexecutive agent may be trained for performing actions on behalf of orrecommending actions to a user in an executive role. In some of theseembodiments, the client application 8052 may provide the functionalityof the enterprise management platform 8000. For example, in someembodiments, users may view executive digital twins and/or may use thecollaboration tools via the client application 8052. During the use ofthe client application 8052, an executive may either escalate issuesidentified in the respective executive digital twin to another member ofthe enterprise. Each time the user interacts with the client application8052, the client application 8052 may monitor the user's actions and mayreport the actions back to the expert agent system 8008. Over time, theexpert agent system 8008 may learn how the particular user responds tocertain situations. For instance, if the user is the CFO and each time acritical state with revenue or costs is identified in the CFO digital,the CFO escalates the critical state to the CEO, the expert agent system8008 may learn to automatically escalate critical revenue states andcritical cost states to the CEO. Further implementations of the expertagent system 8008 are discussed further in the disclosure.

In embodiments, the artificial intelligence services system 8010performs machine learning, artificial intelligence, and analytics taskson behalf of the EMP 8000. In embodiments, the artificial intelligenceservices system 8010 includes a machine learning system that trainsmachine learned models that are used by the various systems of the EMP8000 to perform some intelligence tasks, including robotic processautomation, predictions, classifications, natural language processing,and the like. In embodiments, the EMP 8000 includes an artificialintelligence system that performs various AI tasks, such as automateddecision making, robotic process automation, and the like. Inembodiments, the EMP 8000 includes an analytics system that performsdifferent analytics across enterprise data to identify insights tovarious states of an enterprise. For example, in embodiments, theanalytics system may analyze the financial data of an enterprise todetermine whether the enterprise is financially stable, in a criticalcondition, or a desirable condition. In embodiments, the analyticssystem may perform the analytics in real-time as data is ingested fromthe various data sources to update one or more states of an enterprisedigital twin. In embodiments, the intelligence system includes a roboticprocess automation system that learns behaviors of respective users andautomates one or more tasks on behalf of the users based on the learnedbehaviors. In some of these embodiments, the robotic process automationsystem may configure expert agents on behalf of an enterprise. Therobotic process automation system may configure machine-learned modelsand/or AI logic that operate to output actions given stimulus. Inembodiments, the robotic process automation system receives trainingdata sets of interactions by experts and configures the machine-learnedmodels and/or AI logic based on the training data sets. In embodiments,the artificial intelligence services system 8010 includes a naturallanguage processing system that receives text/speech and determines acontext of the text and/or generates text in response to a request togenerate text. The intelligence services are discussed in greater detailthroughout the disclosure.

In embodiments, the EMP 8000 includes an enterprise data store 8012 thatstores data on behalf of customer enterprises. In embodiments, eachcustomer enterprise may have an associated data lake that receives datafrom various data sources 8020. In some embodiments, the EMP 8000receives the data via one or more APIs 8014. For example, inembodiments, the API may be configured to obtain real-time sensor datafrom one or more sensor systems 8022 of an enterprise. The sensor datamay be collected in a data lake associated with the enterprise. Thedigital twin system 8004 and the artificial intelligence services system8010 may structure the data in the data lake and may populate one ormore respective enterprise digital twins based on the collected data. Insome embodiments, the data sources 8020 may include a set of edgedevices 8042 that collect, receive and process data from the sensorsystem 8022, from suitable IoT devices, from local networking devices(e.g., wireless and fixed network resources, including repeaters,switches, mesh network nodes, routers, access points, gateways, andothers), from general purpose networking devices (e.g., computers,laptops, tablets, smartphones and the like), from smart products, fromtelemetry systems of machinery, equipment, systems and components (e.g.,onboard diagnostic systems, reporting systems, streaming systems,syndication systems, event logs and the like), data collected by datacollectors (including drones, mobile robots, RFID and other readers, andhuman-portable collectors) and/or other suitable data sources. In someof these embodiments, the edge devices 8042 may be configured to processsensor data (or other suitable data) collected at a “network edge” ofthe enterprise. Edge processing of enterprise data may include sensorfusion, data compression, computation, filtering, aggregation,multiplexing, selective switching, batching, packetization, streaming,summarization, fusion, fragmentation, encoding, decoding, transcoding,copying, storage, decompression, syndication, augmentation (e.g., bymetadata), content inspection, classification, extraction,transformation, normalization, loading, formatting, error correction,data structuring, and/or many other processing actions. In someembodiments, the edge device 8042 may be configured to operate on thecollected data and to adjust an output data stream or feed based on thecontents of the collected data and/or based on contextual information,such as network conditions, operational conditions, environmentalconditions, workflow conditions, entity state information, datacharacteristics, or many others. For example, an edge device 8042 maystream granular sensor data that is identified to be anomalous withoutcompression, while the edge device 8042 may compress, summarize, orotherwise pass on a less granular data that is considered to be within atolerance range of normal conditions or that reflects characteristics(e.g., statistical or signal characteristics) that suggest a lowerlikelihood that the data is likely to be of high interest. In this way,the edge device 8042 may provide semi-sentient data streams.Semi-sentience at the edge device 8042 may be improved by machinelearning and training on a set of outcomes or feedback from users usingprocess automation, machine learning, deep learning, or other artificialintelligence techniques as described herein. In embodiments, the EMP8000 may store the data streams in the data lake and/or may update oneor more enterprise digital twins with some or all of the received data.

In embodiments, the client devices 8050 may execute one or more clientapplications 8052 that interface with the EMP 8000. In embodiments, aclient application 8052 may request and display one or more enterprisedigital twins. In some of these embodiments, a client application 8052may depict an executive digital twin corresponding to the role of theuser. For example, if the user is designated as the Chief MarketingOfficer, the EMP 8000 may provide a CMO digital twin of the enterpriseof the user. In some of these embodiments, the user data stored at theEMP 8000 and/or the client device 8050 may indicate the role of the userand/or the types of enterprise digital twins (and features thereof) towhich the user has access.

In embodiments, the client application 8052 may display the requestedexecutive digital twin and may provide one or more options to performone or more respective actions/operations corresponding to the executivedigital twin and the states depicted therein. In embodiments, theactions/operations may include one or more of “drilling down” into aparticular state, escalating or otherwise notifying another user of astate or set of states, exporting a state or set of states into acollaborative environment (e.g., into a word processor document, aspreadsheet, a presentation document, a slide show, a model (e.g., a CADmodel, a 3D model, or the like), a report (e.g., an annual report, aquarterly report, or the like), a website, a Wiki, a dashboard, acollaboration environment location (e.g., a Slack™ location), a workflowapplication, or the like), sending a request for action with respect toone or more states from another user, performing a simulation, adjustinginterface elements (such as changing sizes, colors, locations,brightness, presence/absence of display, etc.), or the like. Forexample, a COO or other operations executive may view an operations orCOO digital twin. The states that may be depicted in the COO digitaltwin may include notifications of potential issues with one or morepieces of machinery or equipment (e.g., among many others, as observedfrom analyzing a stream of data from one or more sensors on a piece ofrobotic equipment). In viewing the COO digital twin, the user may wishto escalate the issue, such as to the CEO, request input from anotherexecutive and/or to instruct an operations manager, such as a warehouseor plant manager, to handle the issue. In this example, the clientapplication depicting the COO digital twin may allow the user to selectan option to escalate the issue. In response to the user selecting the“escalate” option, the client application 8052 transmits the escalaterequest to the EMP 8000. The EMP 8000 may then determine the appropriateuser or users to which the issue is escalated. In some embodiments, theEMP 8000 may determine the reporting structure of the enterprise from anorganizational digital twin of the enterprise to which the users belong.In this example, if the operations executive elects to have theoperations manager handle the issue, the user may select an option toshare the state with another user. The user may then enter an identifierof the intended recipient (e.g., an email address, phone number, textaddress, user name, role description, or other identifier of therecipient (such as identifiers for the recipient in various workflowenvironments, collaboration environments and the like (including otherdigital twins), and the like) and may input a message indicatinginstructions to the intended recipient. In response, the EMP 8000 maycommunicate the identified state to the intended recipient.

In another example, the client application 8052 may depict a CFO digitaltwin to a user (e.g., the CFO of an enterprise). In this example, theCFO may be tasked with preparing a quarterly report at the request ofthe CEO. In this example, the CFO may view a set of different financialstates, including a P&L data, historical sales data (e.g., quarterlysales data and/or annual sales data), real-times sales data, projectedsales data, historical cost data (e.g., quarterly costs and/or annualcosts), projected costs, and the like. In this example, the CFO mayselect the states to include in the annual report, including the P&Ldata, quarterly sales data, and quarterly cost data. In response to theuser selection, the client application 8052 may transmit a request toexport the selected states into the annual report. In this example, theEMP 8000 may receive the request, identify the document (e.g., theannual report), and may include the selected states into the identifieddocument.

In embodiments, the client application 8052 may include a monitoringagent that monitors the manner by which a user responds to specificrequests (e.g., a request from the CEO to populate a report) ornotifications (e.g., a notification that a piece of machinery requiresmaintenance). The monitoring agent may report the user's response tosuch prompts to the EMP 8000. In response, the EMP 8000 may train anexecutive agent (which may include one or more machine-learned models)to handle such notifications when they next arrive. In some embodiments,the monitoring agent may be incorporated in an executive agent that isincorporated in the client application 8052.

FIG. 69 illustrates an example set of components of a digital twinsystem 8004. As discussed, a digital twin system 8004 is configured togenerate visual and/or data-based digital twins, including enterprisedigital twins, and to serve the digital twins to a client (e.g., a userdevice, a server, and/or internal and/or external applications thatleverage digital twins). In embodiments, the digital twin system 8004 isan infrastructure component of the EMP 8000. In embodiments, the digitaltwin system 8004 is a microservice that is accessible by the EMP 8000and/or other components of a value chain control tower.

In embodiments, the digital twin system 8004 is executed by a computingsystem (e.g., one or more servers) that may include a processing system8100 that includes one or more processors, a storage system 8120 thatincludes one or more computer-readable mediums, and a network interface8130 that includes one or more communication units that communicate witha network (e.g., the Internet, a private network, and the like). In theillustrated example embodiments, the processing system 8100 may executeone or more of a digital twin configuration system 8102, digital twinI/O system 8104, a data structuring system 8106, a digital twingeneration system 8108, a digital twin perspective builder 8110, adigital twin access controller 8112, a digital twin interaction manager8114, an digital twin simulation system 8116, and a digital twinnotification system 8118. The processing system 8100 may executeadditional or alternative components without departing from the scope ofthe disclosure. In embodiments, the storage system 8120 may storeenterprise data, such as an enterprise data lake 8122, a digital twindata store 8124, a behavior datastore 8126 and/or other datastore, suchas a distributed datastore, such as a set of blockchains or distributeddata storage resources. The storage system 8120 may store additional oralternative data stores without departing from the scope of thedisclosure. In embodiments, the digital twin system 8004 may interfacewith the other components of the EMP 8000, such as the enterpriseconfiguration system 8002, the collaboration suite 8006, the expertagent system 8008, and/or the artificial intelligence services system8010.

In embodiments, the digital twin configuration system 8102 is configuredto set up and manage the enterprise digital twins and associatedmetadata of an enterprise, to configure the data structures and datalistening threads that power the enterprise digital twins, and toconfigure features of the enterprise digital twins, including accessfeatures, processing features, automation features, reporting features,and the like, each of which may be affected by the type of enterprisedigital twin (e.g., based on the role(s) that it serves, the entities itdepicts, the workflows that it supports or enables and the like). Inembodiments, the digital twin configuration system 8102 receives thetypes of digital twins that will be supported for the enterprise, aswell as the different objects, entities, and/or states that are to bedepicted in each type of digital twin. For each type of digital twin,the digital twin configuration system 8102 determines one or more datasources and types of data that feed or otherwise support each object,entity, or state that is depicted in the respective type of digital twinand may determine any internal or external software requests (e.g., APIcalls) that obtain the identified data types or other suitable dataacquisitions mechanisms, such as webhooks, that are configured toautomatically receive data from an internal or external data source Insome embodiments, the digital twin configuration system 8102 determinesinternal and/or external software requests that support the identifieddata types by analyzing the relationships between the different types ofdata that correspond to a particular state/entity/object and thegranularity thereof. Additionally or alternatively, a user may define(e.g., via a GUI) the data sources and/or software requests and/or otherdata acquisition mechanisms that support the respective data types thatare depicted in a respective digital twin. In these embodiments, theuser may indicate the data source that are to be accessed and the typesof data to be obtained from the respective data source. For example, ifa user is configuring an enterprise digital twin of a supply chainprocess, the user may identify an inventory management system to obtaininventory levels, various supplier systems to obtain pricing data ofparticular items, sensor systems to obtain sensor data from variouspoints within the enterprise's supply chain (e.g., manufacturingfacilities, warehouse facilities, and the like), and other suitablesystems for other suitable data types. In this data definition process auser may associate specific data types and/or data sources tocorresponding structural elements of a digital twin (e.g., layouts,spatial elements, processes, or components thereof). For example, theuser can match a specific cost of a good (e.g., the cost of a bearing ona compressor, a headlight that goes into an automobile, an automobile,or any other suitable good) that is obtained via an API request to aseller of the good with a digital twin element representing the good(e.g., a 3D model of the good). In this example, the digital twin of thegood may depict the cost of the good, and as the price of the goodchanges, so too may the depiction of the good.

In embodiments, the configuration system 8102 generates one or moreforeign keys for each digital twin that collectively associate differentdata types with the structural elements of the digital twin. Thus, whena digital twin is generated, the foreign key may be leveraged to connectdata obtained from the data sources to the structural elements of thedigital twin. In some embodiments, a configuring user may define theassociations that are used to generate the set of foreign keys.

In embodiments, the digital twin configuration system 8102 determines,defines, and manages the data structures needed to support each type ofdigital twin, such as data lakes, relational databases, SQL databases,NOSQL databases, graph databases, and the like. For example, for anenvironment digital twin, the digital twin configuration system 8102 mayinstantiate a database (e.g., a graph database that defines the ontologyof the environment and the objects existing (or potentially existing)within the environment and the relationships therebetween), whereby theinstantiated database contains and/or references the underlying datathat powers the environmental digital twin (e.g., sensor data andanalytics relating thereto, 3D maps, physical asset twins within theenvironment, and the like). In some embodiments, a user may define anontology of a respective digital twin, such that the ontology definesthe types of data depicted in the digital twin and the relationshipsbetween those data types. Additionally or alternatively, the digitaltwin configuration system 8102 may derive the ontology based on thetypes of digital twins that are to be configured.

In some embodiments, the different types of enterprise digital twins maybe configured in accordance with a set of preference settings,granularity settings, alert settings, taxonomy settings, topologysettings, and the like. In some embodiments, the configuration system8102 may utilize pre-defined preferences (e.g., default preferencetemplates for different types of enterprise digital twins, includingones that are domain-specific, role-specific, industry-specific,workflow-specific and the like), taxonomies (e.g., default taxonomiesfor different types of enterprise digital twins), and/or topologies(e.g., default topologies for different types of twins, such asgraph-based topologies, tree-based topologies, serial topologies,flow-based topologies, loop-based topologies, network-based topologies,mesh topologies, and others)). Additionally or alternatively, theconfiguration system 8102 may receive custom preference settings andtaxonomies from a configuring user. Non-limiting examples ofrole-specific templates that are used to configure a role-based digitaltwin may include may include CEO template, a COO template, a CFOtemplate, a counsel template, a board member template, a CTO template, achief marketing officer template, an information technology managertemplate, a chief information officer template, a chief data officertemplate, an investor template, a customer template, a vendor template,a supplier template, an engineering manager template, a project managertemplate, an operations manager template, a sales manager template, asalesperson template, a service manager template, a maintenance operatortemplate, and/or a business development template. Similarly, examples oftaxonomies that are used to configure different types of role-baseddigital twins may include CEO taxonomy, a COO taxonomy, a CFO taxonomy,a counsel taxonomy, a board member taxonomy, a CTO taxonomy, a chiefmarketing officer taxonomy, an information technology manager taxonomy,a chief information officer taxonomy, a chief data officer taxonomy, aninvestor taxonomy, a customer taxonomy, a vendor taxonomy, a suppliertaxonomy, an engineering manager taxonomy, a project manager taxonomy,an operations manager taxonomy, a sales manager taxonomy, a salespersontaxonomy, a service manager taxonomy, a maintenance operator taxonomy,and/or a business development taxonomy. Each of the role-specifictemplates may include data types that are specific to the kinds ofinteractions the role might have and the specific responses tointeractions, which may be role-based. For example, a CEO template mayinclude data type definitions for supplier information and labor costinformation across the entire organization, and may include responses tointeractions with a CEO digital twin, such as drilling down to specificsuppliers and/or labor groups within the enterprise.

In embodiments, the digital twin configuration system 8102 may beconfigured to configure and instantiate the databases that support eachrespective enterprise digital twin of an enterprise (e.g., role-baseddigital twins, environment digital twins, organizational digital twins,process digital twins, and the like), which may be stored on the digitaltwin data store 8124. In embodiments, for each database configuration,the digital twin configuration system 8102 may identify and connect anyexternal resources needed to collect data for each respective data type.For each identified external resource, the digital twin configurationsystem 8102 may configure one or more data collection threads to accessan API, SDK, port, webhook, search facility, database access facility,and/or other connection facility For example, certain executive digitaltwins (e.g., CEO digital twin, CFO digital twin, COO digital twin, andCMO digital twin) may each require data derived and/or obtained from theCRM 8026 of the enterprise. In this example, the digital twinconfiguration system 8102 may configure one or more data collectionthreads to access an API, SDK, port, webhook, search facility, databaseaccess facility, and/or other connection facility of the CRM 8026 of theenterprise on behalf of the enterprise and may obtain any necessarysecurity credentials to access the API. In another example, in order tocollect data from one or more edge devices 8042 of the enterprise, theconfiguration system 8102 may initiate a process of granting access tothe edge devices 8042 of the enterprise to the APIs of the EMP 8000,such that the edge devices 8042 may provide digital twin data to the EMP8000.

In embodiments, the digital twin I/O system 8104 is configured to obtaindata from a set of data sources (e.g., users, sensor systems, internaland/or external databases, software platforms (e.g., CRMs, ERPs, CRMs,workflow management system), surveys, customers, and the like). In someembodiments, the digital twin I/O system 8104 (or other suitablecomponent) may provide a graphical user interface that allows a useraffiliated with an enterprise to upload various types of data that maybe leveraged to generate the enterprise digital twins of the enterprise.For example, in providing data to support an environment digital twin, auser may upload 3D scans, still and video images, LIDAR scans,structured light scans, blueprints, 3D floor plans, object types (e.g.,products, sensors, machinery, furniture, and the like), objectproperties (e.g., materials, physical properties, descriptions, price,and the like), output type (e.g., sensor units), architectural drawings,CAD documents, equipment specifications, and many others via the digitaltwin I/O system 8104. In embodiments, the digital twin I/O system 8104may subscribe to or otherwise automatically receive data streams (e.g.,publicly available data streams, such as RSS feeds, news streams, eventstreams, log streams, sensor system streams, and the like) on behalf ofan enterprise. Additionally or alternatively, the digital twin systemI/O system 8104 may periodically query and/or receive data from aconnected data source 8020, such as the sensor system 8022 havingsensors that sensor data from facilities (e.g., manufacturingfacilities, shipping facilities, warehouse facilities, logisticsfacilities, retail facilities, distribution facilities, agriculturalfacilities, resource extraction facilities, computing facilities,transportation facilities, infrastructure facilities, networkingfacilities, data center facilities, and many others) and/or otherphysical entities of the enterprise, the sales database 8024 that isupdated with sales figures in real time, the CRM system 8026, thecontent marketing platform 8028, financial databases 8030, surveys 8032,org charts 8034, workflow management systems 8036, third-party datasources 8038, customer databases 8040 that store customer data, and/orthird-party data sources 8038 that store third-party data, edge devices8042 that report data relating to physical assets (e.g., smartmachinery/manufacturing equipment, sensor kits, autonomous vehicles, ofthe enterprise, wearable devices, and the like), enterprise resourcemanagement systems 8044, HR systems 8046, content management systems8026, and the like). In embodiments, the digital twin I/O system 8104may employ a set of web crawlers to obtain data. In embodiments, thedigital twin I/O system 8104 may include listening threads that listenfor new data from a respective data source. In embodiments, the digitaltwin I/O system 8104 may be configured with a set of webhooks thatreceive data from a respective set of data sources. In theseembodiments, the digital twin I/O system 8104 may receive data that ispushed from an external data source, such as real-time data.

In some embodiments, the digital twin I/O system 8104 is configured toserve the obtained data to instances of enterprise digital twins (whichis used to populate digital twins) that are executed by the clientdevice 8050 or the EMP 8000. In embodiments, the digital twin I/O system8104 receives data stream feeds received data streams received and/orcollected on behalf on an enterprise and stores at least a portion ofthe streams into a data lake 8122 associated with the enterprise. Inembodiments, the data that is streamed into the data lake 8122 may bestructured and stored in one or more databases stored in the digitaltwin data stores 8124.

In embodiments, the data structuring system 8106 is configured toprocess and structure data into a format that can be consumed by anenterprise digital twin. In embodiments, processing by the datastructuring system 8106 may include compression, computation, filtering,aggregation, multiplexing, selective switching, batching, packetization,streaming, summarization, fusion, fragmentation, encoding, decoding,transcoding, encryption, decryption, duplication, deduplication,normalization, cleansing, identification, copying, storage,decompression, syndication, augmentation (e.g., by metadata), contentinspection, classification, extraction, transformation, loading,formatting, error correction, data structuring, and/or many otherprocessing actions. In embodiments, the data structuring system 8106 mayleverage ETL (extract, transform, load) tools, data streaming, and otherdata integration tooling to structure the various types of digital twindata. In embodiments, the data structuring system 8106 structures thedata according to a digital twin data model that may be defined by thedigital twin configuration system 8102 and/or a user. In embodiments, adigital twin data model may refer to an abstract model that organizeselements of enterprise-related data and standardizes the manner by whichthose elements relate to one another and to the properties of digitaltwin entities. For instance, a digital twin data model of an environmentthat includes vehicles (e.g., a vehicle assembly facility or anenvironment where vehicles operate) may specify that the data elementrepresenting a vehicle be composed of a number of other elements whichrepresent sub-elements or attributes of the vehicle (the color of thevehicle, the dimensions of the vehicle, the engine of the vehicle, theengine parts of the vehicle, the owner of the vehicle, the performancespecifications of the vehicle, and the like). In this example, thedigital twin model components may define how the physical attributes aretied to respective physical locations on the vehicle. In embodiments,digital twin data models may define a formalization of the objects andrelationships found in a particular application domain. For example, adigital twin data model may represent the customers, products, andorders found in a manufacturing enterprise and how they relate to eachother within the various digital twins. In another example, a digitaltwin data model may define a set of concepts (e.g., entities,attributes, relations, tables, and/or the like) used in defining suchformalizations of data or metadata within the environment. For example,a digital twin data model used in connection with a banking applicationmay be defined using the entity-relationship data model and how theentity-relationship data model is then related to the various executivedigital twin views.

In embodiments, the digital twin generation system 8108 servesenterprise digital twins on behalf of an enterprise. In some instances,the digital twin generation system 8108 receives a request for aspecific type of digital twin from a client application 8052 beingexecuted by the client device 8050 (e.g., via an API). Additionally oralternatively, the digital twin generation system 8108 receives arequest for a specific type of digital twin from a component of EMP 8000(e.g., the digital twin simulation system 8116). The request mayindicate the enterprise, the type of digital twin, the user (whoseaccess rights may be verified or determined by the digital accesscontroller 8112), and/or a role of the user. In some embodiments, thedigital twin generation system 8108 may determine and provide the clientdevice 8050 (or requesting component) with the data structures,definition of grain of data the, response patterns to specific inputs,animation sequences for illustrating behaviors, display aggregationmethods for smaller displays (such as mobile phone), immersive datainteraction systems, security constraints on the data viewing, viewinginteraction speed (frame rate), nature of light sources (simulate actualor continuous), multiple user engagement protocols, network bandwidthconstraints, metadata, ontology and information on hooks to data feedsas well as the digital twin constructs. This information may be used bythe client to generate the digital twin in the end user device (e.g., animmersive device, such as AR devices or VR devices, tablet, personalcomputer, mobile, or the like). In embodiments, the digital twingeneration system 8108 may determine the appropriate perspective for therequested digital twin (e.g., via the digital twin perspective builder8110, which may include device-sensitive perspectives, such asdelivering in appropriate formats based on the type of end user device)and any data restrictions, interaction restrictions, depth of datarestrictions, usage restrictions, length of visibility restrictions,that the user may have (e.g., via the access controller 8112). Inresponse to determining the perspective and data restrictions, thedigital twin generation system 8108 may generate the requested digitaltwin. In some embodiments, generating the requested digital twin mayinclude identifying the appropriate data structure given the perspectiveand obtaining the data that parameterizes the digital twin, as well asany additional metadata that is served with the enterprise digital twin.

In embodiments, the digital twin generation system 8108 may deliver theenterprise digital twin to the requesting client application 8052 (orrequesting component). In embodiments, the digital twin generationsystem 8108 (or another suitable component) may continue to update aserved digital twin with real-time data (or data that is derived fromreal-time data) as the real-time data is received and potentiallyanalyzed, extrapolated, derived, predicted, and/or simulated by the EMP8000.

In some embodiments, the digital twin generation system 8108 (incombination with the digital twin I/O system 8104) may obtain datastreams from traditional data sources, such as relational databases, APIinterfaces, direct sensor input, human generated input, Hadoop filestores, graph databases that underlie operational and reporting toolingin the environment, telemetry data sources, onboard diagnostic systems,blockchains, distributed ledgers, distributed data sources, feed,streams, and many other sources. In embodiments, the digital twingeneration system 8108 may obtain data streams that are associated withthe structural aspects of the data, such as the layout and 3D objectproperties of entities within facilities, geospatial informationsystems, the hierarchical design of a system of accounts, and/or thelogical relationships of entities and actions in a workflow. Inembodiments, the data streams may include metadata streams that areassociated with the nature of the data and data streams containingprimary data (e.g., sensor data, sales data, survey data, and the like).For example, the metadata associated with a physical facility or otherentity may include the types and layers of data that are being managed,while the primary data may include the instances of objects that fallwithin each layer. Layers for which metadata may be tracked and/orcreated may include, for example, metadata with respect to attributes,parameters or representations of a whole facility, component systems andassets within the facility (equipment, network entities, workforceentities, assets, and the like), sub-components and sub-systems, andfurther sub-components and sub-systems down to arbitrarily lower levelsof granularity (e.g., a ball bearing of a rotating axle assembly of afan that is part of a motor assembly driving an assembly line in alocation of a warehouse). In embodiments, layers may include, in anotherexample, logical or operational layers, such as a reporting structure,such as from a COO to a VP of operations to a distribution manager to awarehouse manager to a shift manager to a warehouse worker. Inembodiments, layers may include workflow or process flow layers, such asfrom an overall process to its sub-components and decision points, suchas an overall assembly process having sub-layers of gathering of inputmaterials and components, positioning of workers, a series of assemblysteps, inspection of outputs, and delivery to a post-assembly location.

In embodiments, the digital twin perspective builder 8110 leveragesmetadata, artificial intelligence, heuristic methods, 3D renderingalgorithms and/or other data processing techniques to produce adefinition of information required for generation of the digital twin inthe digital twin generation system 8108. In some embodiments, differentrelevant datasets are hooked to a digital twin (e.g., an executivedigital twin, an environment digital twin, or the like) at theappropriate level of granularity, thereby allowing for the structuralaspects of the data (e.g., system of accounts, sensor readings, salesdata, or the like) to be a part of the data analytics process. Oneaspect of making a perspective function is that the user can change thestructural view or the granularity of data while potentially forecastingfuture events or changes to the structure to guide control of the areaof the business at question. In embodiments, the term “grain of data”may refer to the base unit of a type of data, such as a single line ofdata, a single aggregated line of data, a single byte of data, a singlefile, a single instance, or the like. Examples of “grains of data” mayinclude a detailed record on a single sale, a single block in ablockchain in a distributed ledger, a single event in an event log, asingle vibration reading from a vibration sensor, or similar singular oratomic data units, and the like. Grain or atomicity may impose aconstraint in how the data can be combined or processed to formdifferent outputs. For example if some element of data is captured onlyat the level of once-per-day, then it can only be broken down to singledays (or aggregation of days) and cannot be broken down to hours orminutes, unless derived from the day representation (e.g., usinginference techniques and/or statistical models). Similarly, if data isprovided only at the aggregate business unit level, it can be brokendown to the level of an individual employee only by, for example,averaging, modeling, or inductive functions. Generally, role-based andother enterprise digital twins may often benefit from finer levels ofdata, as aggregations and other processing steps may produce outputsthat are dynamic in nature and/or that relate to dynamic processesand/or real-time decision-making. It is noted that different types ofdigital twins may have different “sized” grains of data. For example,the grains of data that feed a CEO digital twin may be at a highergranularity level than the grains of data that feed a COO digital twin.In some embodiments, however, a CEO may drill down into a state of theCEO digital twin and the granularity for the selected state may beincreased.

In embodiments, the perspective builder 8110 adds relevant perspectiveto the data underlying the digital twin, which is provided to thedigital twin generation system 8108. In embodiments, “perspective” mayrefer to the adjustments to, aggregations of, simplifications of, and/ordetail additions to the ontology of a particular digital twin (e.g., arole-based digital twin) that provide the appropriate ontological viewof the underlying data with the correct types at the appropriategranularity level. For example, a CEO digital twin may link in fuzzydata with markets data and depict the potential impacts of market forceson a simulated digital twin environment for different scenarios. Inanother example, in a CFO level digital twin, the internal financialsystem of accounts may be allocated across the physical structure of thedigital twin providing an ability to understand the relationship betweenrevenue generation, cost allocation, and the structural aspects of thebusiness (e.g., the layout of a factory floor, a warehouse, adistribution center, a logistics facility, an office building, a retaillocation, a container ship, or the like). Continuing this example, theCTO digital twin may include data overlays with current marketinformation on new technologies and linkages therebetween. In thisexample, the CTO digital twin builds in linkages between an impact ofchanging technology platforms and outside information that may be usedfor enhancement of the facility. These different perspectives generatedby the perspective builder 8110 combine with the digital twin simulationsystem 8116 to provide relevant simulations of how scenario-based futurestates might be handled by the facility, the digital twin simulationsystem 8116 provides for, recommendations on how to enhance thedigitally twin represented facility structurally to meet the needs ofthe future states, responses to specific changes in the digital twinenvironment or alterations in the information relating to digital twinsimulate elements. In embodiments, the perspective builder 8110 maybuild perspectives that depict intersections or overlays of operationalstates and entities with information technology states and entities,which may facilitate recognition of opportunities and/or problemsinvolving the interplay and convergence of information technology andoperations technology within the operations of a wide range ofindustries and domains. In further embodiments, the perspective builder8110 may build perspectives that allow for different roles to interactwith the same digital twin while maintaining different perspectives onthe operational states and entities, which allows for these differentroles to have a meaningful interaction while maintaining theirrole-specific perspective. In embodiments, the perspective builder 8110builds a perspective for a digital twin by providing each differentuser/role with a respective diagrammatic view expressed as in thedigital twin where that diagram includes information and structure at alevel relevant to the specific user's role. This user-specific diagramis then connected to the underlying data to provide for the role-baseddigital twin experience.

In embodiments, the digital twin access controller 8112 informs thegeneration system 8108 of specific constraints around the roles of usersable to view the digital twin as well as providing for dynamicallyadjustable digital twins that can adapt to constrain or release views ofthe data or other features specific to each user role. For examplesensitive salary data might be obfuscated from most administrativeemployees when viewing an organizational digital twin, but the CEO maybe granted access to view the salary information directly. Inembodiments, the digital twin access controller 8112 may receive a useridentifier and one or more data types. In response, the digital twinaccess controller 8112 may determine whether the user indicated by theuser identifier has access to the one more data types or other features.In some of these embodiments, the digital twin access controller maylook up the user in the organizational digital twin of the enterprise ofthe user and may determine the user's permissions and restrictions basedthereon. Alternatively, the user's permissions and restrictions may beindicated in a user database. In embodiments, the organizational digitaltwin may, as noted above, be generated automatically, such as by parsingavailable data sources to automatically construct a representation ofthe organization, such as a hierarchical organizational chart, a graphof the organization with nodes representing organizational entities(e.g., workgroups, roles, assets and personnel), links or connectionsindicating relationships (e.g., reporting relationships, lines ofauthority, group affiliations, and the like), and data or metadataindicating other attributes of the entities and relationship, and thelike.

In embodiments, the digital twin interaction manager 8114 manages therelationship between the structural view of the data in an enterprisedigital twin (e.g., as depicted/represented by the client application8052) and the underlying data streams and data sources. In embodiments,this interaction layer makes the digital twin into a window into theunderlying data streams through the lens of the structure of the data.In embodiments, the digital twin interaction manager 8114 determines thetypes of data, or the nature of the human interface for building theseinteractions, that are being fed to an instance of an enterprise digitaltwin (e.g., an environment digital twin or an executive digital twin)while the instance is being executed by a client application 8052. Putanother way, the digital twin interaction manager 8114 determines andserves data for an in-use digital twin. In embodiments, the digital twininteraction manager 8114 has specific user interactions and controlsthat govern the relationship between a user interface and the role baseddigital twin. Furthermore, in embodiments, these role-based digital twininteractions can be with a shared digital twin with different rolesinteracting seamlessly. In embodiments, the digital twin interactionmanager 8114 feeds raw data received from a data source to the digitaltwin or from the digital twin I/O system 8104, or a combination of thedigital twin I/O system 8104 and role-based human interactions Forexample, sensor readings of temperatures throughout an environment maybe fed directly to the executing environment digital twin of theenvironment through the digital twin I/O system 8104 and in response toa human interaction with the environment digital twin to adjust atemperature setting of the environment, the digital twin interactionmanager 8114 may issue a control signal to a temperature controllerwithin the environment to increase or decrease the temperature.

In embodiments, the digital twin interaction manager 8114 obtains dataand/or instructions that are derived by another component of the EMP8000. For example, a CEO digital twin may depict analytical dataobtained from the artificial intelligence services system 8010 that isderived from incoming financial data, marketing data, operational data,and sensor data. In this example, the digital twin interaction manager8114 may receive a request to drill down into the analytical data fromthe user and in response, the digital twin interaction manager 8114 mayobtain the financial data, marketing data, and/or the sensor data fromwhich the analytical data was derived. In another example, the digitaltwin interaction manager 8114 may receive simulated cost data from thedigital twin simulation system 8116 to convey revenue/costs with respectto different asset maintenance schedules, whereby the simulated data isderived using historical maintenance data of the enterprise, historicalsensor data collected by sensors in a facility of the enterprise. Inthis example, the digital twin interaction manager 8114 may receiverequests for different maintenance schedules from a client devicedepicting an executive digital twin (e.g., a CFO digital twin, a CTOdigital twin, or a CEO digital twin) and may initiate the simulationsfor each of the different maintenance schedules. The digital twininteraction manager 8114 may then serve the results of the simulation tothe requesting client application.

In embodiments, the digital twin interaction manager 8114 may manage oneor more workflows that are performed via an executive digital twin. Forexample, the EMP 8000 may store a set of executive workflows, where eachexecutive workflow corresponds to a role within an enterprise andincludes one or more stages. In embodiments, the digital twininteraction manager 8114 may receive a request to execute a workflow.The request may indicate the workflow and a user identifier. Inresponse, the digital twin interaction manager 8114 may retrieve therequested workflow and may provide specific instructions, includingrole-based interactions, and/or data to the client device 8052

In embodiments, the digital twin simulation system 8116 receivesrequests to run simulations using one or more digital twins. Inembodiments, the request may indicate a set of parameters that are to bevaried and/or one or more simulation outcomes to output. In embodiments,the digital twin simulation system 8116 may request one or more digitaltwins from the digital twin generation system 8108 and may varying a setof different parameters for the simulation. In embodiments, the digitaltwin simulation system 8116 may construct new digital twins and new datastreams within existing digital twins. In embodiments, the digital twinsimulation system 8116 may perform environment simulations and/or datasimulations. The environment simulation is focused on simulation of thedigital twin ontology rather than the underlying data streams. Inembodiments, the digital twin simulation system 8116 generates simulateddata streams appropriate for respective digital twin environments. Thissimulation allows for real world simulations of how a digital twin willrespond to specific events such as changes in the cost of good supplied,or changes in the demand on the output of the facility.

In embodiments, the digital twin simulation system 8116 implements a setof models, in some instances including role-specific response patterns,(e.g., physical mathematical forecasts, logical representations, orprocess diagrams) that develop the framework where data and the responseof the digital twin can be simulated in response to differentsituational or contextual inputs/stimuli. In embodiments, the digitaltwin simulation system 8116 may include or leverage a computerized modelbuilder that constructs a predicted future state of either the dataand/or the response of the digital twin to the input data. In someembodiments, the computerized model library may be obtained from abehavior model data store 8126 that stores one or more models thatdefines one or more behaviors of entities, such as based on scientific,economic, statistical, psychological, sociological, econometric,engineering, mathematical, physical, chemical, biological,architectural, computational, or other models, formulas, functions,processes, algorithms, or the like of the various types described hereinor in the documents incorporated by reference herein (collectivelyreferred to herein as “behavior models” or “models” except where contextindicates otherwise). In embodiments, value chain network data objectsmay be provided according to an object-oriented data model that definesclasses, objects, attributes, parameters and other features of the setof data objects (such as associated with value chain network entitiesand applications) that are handled by the platform. The computerizeddigital twin model calculates the results of the model based onavailable inputs to build an interactive environment where users canwatch and manipulate salient features of the simulated environmentseeing how the entire system responds to specific changes in theenvironment. For example, the digital twin simulation may display how aset of objects that are stacked in a container will respond to tiltingthe container, where the behavior of the objects is based on amechanical engineering model and/or an architectural model of thestacked objects, including structural features, weight distributions,and the like. This may assist in assessing the probability and/or impactof various fault modes, such as breaking, spilling, or the like, inresponse to seismic events, road conditions, weather conditions, waveaction, or the like, as well as in simulating the response of otherobjects in the simulated environment, including in a chain of events.This may, for example, allow a user to identify events and consequencesthat occur as a result of multiple simultaneous or related faults orother events.

In embodiments, digital twin behavior models may be updated and improvedusing results of actual experiments and real-world events. The use ofsuch digital twin mathematical models and their simulations avoidsactual experimentation, which can be costly and time-consuming. Instead,acquired knowledge about behavior of entities and computational powerare used to diagnose and solve real-world problems cheaply and/or in atime-efficient manner. As such, the digital twin simulation system 8116can facilitate understanding a system's behavior without actuallytesting the system in the real world. For example, to determine whichtype of wheel configuration would improve traction the most whiledesigning a tractor, a digital twin model simulation of the tractorcould be used to estimate the effect of different wheel configurationson towing capacity. Useful insights about different decisions in thedesign may be gleaned without actually building the tractor. Inaddition, the digital twin simulation can support experimentation thatoccurs totally in software, or in human-in-the-loop environments wherethe digital twin represents systems or generates data needed to meetexperiment objectives. Furthermore, digital twin simulations can be usedto train persons using a perspective-appropriate virtual environmentthat would otherwise be difficult or expensive to produce.

In embodiments, simulation environments may be constructed using modelsconfigured to predict a set of future states. These models may includedeep learning, regression models, quantum prediction engines, inferenceengines, pattern recognition engines, and many other forms of modellingengines that use historical outcomes, current state information, andother inputs to build a future state prediction. In some embodiments, aconsideration in making the digital twin models' function is the abilityto also show the response of the perspective-based digital twinstructural elements (e.g., defining the deformation of the axle of avehicle in response to different size loads). For example, the resultantdigital twin representation can then be presented to the user in avirtual reality or augmented reality environment where specificperspectives are shown in their digital twin form.

In embodiments, digital twins, as described herein, may operate incoordination with an adaptive edge computing system and/or a set ofadaptive edge computing systems that provide coordinated edgecomputation include a wide range of systems, such as classificationsystems (such as image classification systems, object type recognitionsystems, and others), video processing systems (such as videocompression systems), signal processing systems (such asanalog-to-digital transformation systems, digital-to-analogtransformation systems, RF filtering systems, analog signal processingsystems, multiplexing systems, statistical signal processing systems,signal filtering systems, natural language processing systems, soundprocessing systems, ultrasound processing systems, and many others),data processing systems (such as data filtering systems, dataintegration systems, data extraction systems, data loading systems, datatransformation systems, point cloud processing systems, datanormalization systems, data cleansing system, data deduplicationsystems, graph-based data storage systems, object-oriented data storagesystems, and others), predictive systems (such as motion predictionsystems, output prediction systems, activity prediction systems, faultprediction systems, failure prediction systems, accident predictionsystems, event predictions systems, event prediction systems, and manyothers), configuration systems (such as protocol selection systems,storage configuration systems, peer-to-peer network configurationsystems, power management systems, self-configuration systems,self-healing systems, handshake negotiation systems, and others),artificial intelligence systems (such as clustering systems, variationsystems, machine learning systems, expert systems, rule-based systems,deep learning systems, and many others), system management and controlsystems (such as autonomous control systems, robotic control systems, RFspectrum management systems, network resource management systems,storage management systems, data management systems, and others),robotic process automation systems, analytic and modeling systems (suchas data visualization systems, clustering systems, similarity analysissystems, random forest systems, physical modeling systems, interactionmodeling systems, simulation systems, and many others), entity discoverysystems, security systems (such as cybersecurity systems, biometricsystems, intrusion detection systems, firewall systems, and others),rules engine systems, workflow automation systems, opportunity discoverysystems, testing and diagnostic systems, software image propagationsystems, virtualization systems, digital twin systems, IoT monitoringsystems, routing systems, switching systems, indoor location systems,geolocation systems, and others.

In embodiments, the digital twin notification system 8118 providesnotifications to users via enterprise digital twins associated with therespective users. In some embodiments, digital twin notifications are animportant part of the overall interaction. Digital twin notificationsystem 8118 may provide the digital twin notifications within thecontext of the digital twin setting so that the perspective view of thenotification is set up specifically to enable enlightenment of how thenotification fits into the general digital twin represented ontology,taxonomy, topology or the like.

As discussed, a digital twin model is based on a combination of data andthe data's relationship to the digital twin environments and/orprocesses. As such, different digital twins may share the same data anddifferent digital twin perspectives can be the results of a set ofmetadata built on top of a digital twin data model or data environment.In embodiments, the digital twin data model provides the details of theinformation to be stored and it is used to build a layered system wherethe final computer software code is able to represent the information inthe lower levels in a form that is appropriate for the digital twinperspective being used. One aspect of the digital twin model is that onedigital can be shared across multiple perspectives, each perspectiveviewer can then interact with the same underlying digital twin model. Inthis way the multiple perspectives are like translations allowing eachtype of user to interact in an appropriate way for their skill sets ortheir level of knowledge.

FIG. 70 illustrates an example of a digital twin data model and themanner by which a digital twin is generated, executed, and served to arequesting digital twin application, wherein the digital twin data modeldefines the physical implementation of the underlying data streams fromexisting systems and digital twin structures to achieve a digital twinrepresentation. In embodiments, the digital twin data model 81B00defines the manner by which traditional data streams are tied togetherwith the digital twin structures to achieve the digital twinrepresentation. In embodiments, digital twins are a combination ofprocesses/structures and system data streams. Put another way, processand structure definitions define the real-world “things” (for example afactory, a robot, a cargo container, a ship, a road, or the like) orlogical “things” (for example an organizational chart, a hiring process,a marketing campaign, a tax reporting workflow, or the like) that arerepresentable by a digital twin, while the system data streamdefinitions define the manner by which real-world data may be ingestedinto digital twin representations of the real-world and/or logical“things”. Thus, configuring a digital twin includes structuralconfiguration and ingestion and data configuration and ingestion.

During structural configuration and ingestion, the digital twin system8004 receives the structural aspects of a digital twin. In embodiments,the structural aspects may include process definitions, layoutdefinitions, and/or spatial definitions. In embodiments, a processdefinition defines a logical process that can be mapped to adiagrammatic format that forms the basis of what a digital twin viewercan interact with. Examples of processes may include workflows, hiringprocesses, manufacturing processes, logistics processes, inventoryprocesses, product management processes, software processes, and thelike. In embodiments, the spatial definition defines the geospatialconfiguration of an object or an environment. In embodiments, thespatial definition may be a 2D or 3D representation of an object or anenvironment. The spatial definition of an object or an environment maybe provided as a CAD file, a LIDAR scan, a 2D or 3D image, or the like,including logical relationships, organizational hierarchy, physicalrelationships, schematic relationships, and/or interconnectivity betweenobjects and/or environments. In embodiments, a layout definition definesthe relationship between objects with other objects and/or anenvironment. In embodiments, the layout definition may further definethe manner by which objects move with respect to other objects and/or anenvironment. Examples of layouts may include electrical wiring diagrams,piping schematics, assembly line diagrams, circuit diagrams,hierarchical relationships, network layouts, network schematics,organizational charts, and the like. In embodiments, a layout definitionmay include a set of properties of an object or environment. Examples ofproperties of an object may include physical properties, such as amaterial of an object, a weight of an object, a density of an object, aconductivity of an object, a resistance of an object, a maximum speed ofan object, a maximum acceleration of an object, possible movements of anobject, a reactivity of an object, and/or the like. Examples ofproperties of an environment may include materials of the floors, walls,the roof, and the like, coefficient of friction of the floor, restrictedareas within the environment, paths within the environment, and/or othersuitable properties. In some embodiments, users may upload layoutdefinitions, process definitions, and/or spatial definitions to thedigital twin system 8004. Additionally or alternatively, the digitaltwin system 8004 may provide a graphical user interface that allowsusers to define the layout definitions, process definitions, and/orspatial definitions. In some embodiments, users may import digital twinsfrom 3rd party sources. For example, a producer of a particular objectmay also provide a digital twin of the object, which may then beimported to the digital twin system 8004.

During system data configuration and ingestion, a user defines the datasources that provide data that hydrates or populates a digital twin andconfigures a data bus to receive data from the various data sources. Asdiscussed, the data sources may be received from various systems,including sensor systems, ERPs, CRMs, financial systems, inventorymanagement systems, invoicing systems, 3rd party systems (e.g., weatherservices, news services, government databases, and the like), and othersuitable systems. In embodiments, the user may identify the data sourcesand may provide any information required to enable a data bus to receivedata from the data sources and may further define the associationsbetween the data derived from the data sources and the digital twinelements. A data bus may refer to a middleware layer that provides thedata wiring and data infrastructure for moving data from one system toanother. The data bus may be configured to handle real-time data, nearreal-time data, aggregated data, and/or stored data, or any combinationthereof. The data bus may provide data directly to a digital twin and/ormay store the data in the data warehouse that hydrates the digitaltwins. In embodiments, the user may provide API interface or keys and/orwebhook URLs to the digital twin system 8004 (e.g., via a GUI) therebyenabling data acquisition from the data sources. In embodiments, thedigital twin system 8004 may configure the data bus to access the datasources and/or to receive data from the data sources. In some of theseembodiments, the digital twin system 8004 may generate a webhook URL fora particular digital twin or set of digital twins and may provide thewebhook URL to the data source, such that the data source can pushreal-time or near real-time data to the data bus. Additionally oralternatively, the digital twin system 8004 may obtain an API interfaceor key from the data source, such that the data bus can request datafrom the data source using the API interface or key.

In embodiments, the digital twin system 8004 may generate a foreign keythat associates different types of data with the structural elements ofthe digital twin. In this way, the foreign key ties particular datatypes to various structural or logical or schematic elements, such thatwhen the digital twin is depicted, the real-world data collected fromthe various data sources is connected to the corresponding states of thedigital twin. For example, sensor data received from a subset of sensorsof a sensor system that monitor a particular machine component in a realworld environment may be associated with a digital twin of a machinecomponent, such that the sensor data may be depicted in the digital twinof the machine component. In embodiments, the user may provide input tothe digital twin system 8004 during the configuration phase to tieparticular data types to various elements of a digital twin. The datatypes that are associated with the digital twin may include raw data,processed data, analytical data, derived data, and the like. To theextent a particular data stream is processed before being served into adigital twin (e.g., sensor data that is averaged over a period of timeor a warning condition that is depicted when sales data dips below athreshold), the user may define the operations or the associated displayhighlight that are performed on the data before it is served into adigital twin. In these scenarios, the processed data may be associatedwith a respective digital twin component in the foreign key.

Once the data bus is configured for a particular digital twin and thestructural, logical, or schematic elements (e.g., layout definitions,process definitions, and spatial definitions) of the digital twin aredefined, the digital twin system 8004 may perform digital simulations onthe digital twin and/or may serve the digital twin to a digitaltwin-enabled application based on the structural elements of the digitaltwin, the connected systems data sources, and the foreign key of thedigital twin. In embodiments, the digital twins may be role-baseddigital twin, whereby the views into the digital twin that are served toa user occupying a particular role within an organization. In this way,each user can interact with a respective role-based digital twin and maygain appropriate perspectives based on their respective needs withrespect to an organization. In another embodiment, a plurality of userscan interact with a shared role-enabled digital twin and may gainappropriate perspectives based on their respective needs with respect toan organization to that single digital twin. In embodiments, arole-based digital twin may allow the user to provide feedback to thesource systems to allow for controls of the source system environments,such as corrective actions taken with respect to a source system. Insome embodiments, a plurality of users can make operational changes witha shared role-based digital twin and each user sees these changes in anappropriate way for their role. Furthermore if the operational changeinvolves multiple users, the digital twin can enable a role-basedworkflow management of the depicted environment (e.g., the CEO mayapprove an expenditure to change machinery as requested by the CTO).

In embodiments, the digital twin system 8004 may receive requests toexecute digital twin simulations with respect to a digital twin.Requests to perform digital twin simulations may be received fromdigital twin applications and/or from internal processes. Inembodiments, a digital twin simulation allows for the building ofinteractive models based on the processes, layouts, and/or spatialrepresentations of a digital twin. The digital twin simulations mayprovide the degrees of freedom to allow for the different processes tobe altered in response to dynamic data inputs. For example, a digitaltwin simulation may be executed to depict how a bearing can move on acompressor when the compressor is operated at different operatingconditions or how water flows through a systems of pipes model atdifferent temperatures or with different amounts of buildup in thepiping. In embodiments, the digital twin system 8004 may output theresults of the simulation, which may, for example, depict the impact ofthe simulation parameters on a particular aspect of the digital twin.

In embodiments, a digital twin application may request and depict adigital twin to a user, this digital twin can be a new twin for thatuser or role specific access with role specific views to an existing orshared digital twin. A digital twin application may be provided onmobile applications, virtual reality applications, PCs, and the like. Inembodiments, a digital twin application provides a request to thedigital twin system 8004 for a particular digital twin, where therequest may include a user identifier of the user and/or a role of theuser. In embodiments, the digital twin system 8004 may include orinterface with digital twin application coordinators that receiverequests from digital twin applications for a digital twin. Inembodiments, a digital twin application controller maintains andleverages a set of business rules for a particular digital twin that arerequired by a digital twin application. In some of these embodiments,the set of role-based rules are a set of role-based rules that controlthe states that a user can access given their role within anorganization and a clearance of the user. In these embodiments, thedigital twin application controller may determine whether to grant aninstance of a digital twin application access to a particular user basedon the business rules and the role of the user. In embodiments, thedigital twin system 8004 may include an application services layer thatallows multiple users to connect to the back end of the digital twinapplication coordinator, either directly or through a shared digitaltwin. In embodiments, these connections may include web services,publish and subscribe information buses, simple object access protocols,and/or other suitable application interfaces. The application serviceslayer may return a requested digital twin to a requesting instance of adigital twin application, which in turn depicts the digital twin to theuser. The user may then interact with the digital twin via theapplication to view different states of the digital twin, to requestsimulations, or to interact with other users of the same role ordifferent roles in the digital twin environment, and the like.

In an example implementation of the framework discussed in FIG. 70 , thedigital twin system 8004 may be configured to generate enterprisedigital twins in connection with a value chain. For example, anenterprise that produces goods internationally (or at multiplefacilities) may configure a set of digital twins, such as supplier twinsthat depict the enterprise's supply chain, factory twins of the variousproduction facilities, product twins that represent the products made bythe enterprise, distribution twins that represent the enterprise'sdistribution chains, and other suitable twins. In doing so, theenterprise may define the structural elements of each respective digitaltwin as well as any system data that corresponds to the structuralelements of the digital twin. For instance, in generating a productionfacility twin, the enterprise may the layout and spatial definitions ofthe facility and any processes that are performed in the facility. Theenterprise may also define data sources corresponding to value chainentities, such as sensor systems, smart manufacturing equipment,inventory systems, logistics systems, and the like that provide datarelevant to the facility. The enterprise may associate the data sourceswith elements of the production facility and/or the processes occurringthe facility. Similarly, the enterprise may define the structural,process, and layout definitions of its supply chain and its distributionchain and may connect relevant data sources, such as supplier databases,logistics platforms, to generate respective distribution chain andsupply chain twins. The enterprise may further associate these digitaltwins to have a view of its value chain. In embodiments, the digitaltwin system 8004 may perform simulations of the enterprise's value chainthat incorporate real-time data obtained from the various value chainentities of the enterprise. In some of these embodiments, the digitaltwin system 8004 may recommend decisions to a user interacting with theenterprise digital twins, such as when to order certain parts formanufacturing a certain product given a predicted demand for themanufactured product, when to schedule maintenance on machinery and/orreplace machinery (e.g., when digital simulations on the digital twinindicates the demand for certain products may be the lowest or when itwould have the least effect on the enterprise's profits and lossesstatement), what time of day to ship items, or the like. The foregoingexample is a non-limiting example of the manner by which a digital twinmay ingest system data and perform simulations in order to further oneor more goals.

FIG. 71 illustrates examples of different types of enterprise digitaltwins, including executive digital twins, in relation to the data layer,processing layer, and application layer of the enterprise digital twinframework. In embodiments, executive digital twins may include, but arenot limited to, CEO digital twins 8302, CFO digital twins 8304, COOdigital twins 8306, CMO digital twins 8308, CTO digital twins 8310, CIOdigital twins 8312, GC digital twins 8314, HR digital twins 8316, andthe like. Additionally, the enterprise digital twins that may berelevant to the executive suite may include cohort digital twins 8320,agility digital twins 8322, CRM digital twins 8324, and the like. Thediscussion of the different types of digital twins is provided forexample and not intended to limit the scope of the disclosure. It isunderstood that in some embodiments, users may alter the configurationof the various executive digital twins based on the business needs ofthe enterprise, the reporting structure of the enterprise, and the rolesand responsibilities of the various executives within the enterprise.

In embodiments, executive digital twins and the additional enterprisedigital twins are generated using various types of data collected fromdifferent data sources. As discussed, the data may include real-timedata 8330, historical data 8332, analytics data 8334, simulation/modeleddata 8336, CRM data 8338, organizational data, such as org charts and/oran organizational digital twin 8340, an enterprise data lake 8342, andmarket data 8344. In embodiments, the real-time data 8330 may includesensor data collected from one or more IoT sensor systems, which may becollected directly from each sensor and/or by various data collectiondevices associated with the enterprise, including readers (e.g., RFID,NFC, and Bluetooth readers), beacons, gateways, repeaters, mesh networknodes, WIFI systems, access points, routers, switches, gateways, localarea network nodes, edge devices, and the like. Real-time data 8330 mayinclude additional or alternative types of data that are collected inreal-time, such as real-time sales data, real-time cost data, projectmanagement data that indicates the status of current projects, and thelike. Historical data may be any data collected by the enterprise and/oron behalf of the enterprise in the past. This may include sensor datacollected from the sensor systems of the enterprise, sales data, costdata, maintenance data, purchase data, employee hiring data, employeeon-boarding data, employee retention data, legal-related data indicatinglegal proceedings, patent filing data indicating patent filings andissued patents, project management data indicating historical progressof past and current projects, product data indicating products that areon the market, and the like. Analytics data 8334 may be data derived byperforming one or more analytics processes on data collected by and/oron behalf of the enterprise. Simulation/modeled data 8336 may be anydata derived from simulation and/or behavior modeling processes that areperformed with respect to one or more digital twins. CRM data 8336 mayinclude data obtained from a CRM of the enterprise. An organizationaldigital twin 8340 may be a digital twin of the enterprise. Theenterprise data lake 8342 may be a data lake that includes datacollected from any number of sources. In embodiments, the market data8342 may include data that is collected from disparate data sourcesconcerning or related to competitors and other cohorts in themarketplace and supply chain. Market data 8342 may be collected frommany different sources and may be structured or unstructured. Inembodiments, market data 8342 may contain an element of uncertainty thatmay be depicted in a digital twin that relies on such market data 8342,such as by showing error bars, probability cones, random walk paths, orthe like. It is appreciated that the different types of data highlightedabove may overlap. For example: historical data may be obtained from theCRM data; the enterprise data lake 8342 may include real-time data 8330,historical data 8332, analytics data 8332, simulated/modeled data 8336,and/or CRM data 8336; and analytics data 8334 may be based on historicaldata 8332, real-time data 8332, CRM data 8336, and/or market data 8342.Additional or alternative types of data may be used to populate anenterprise digital twin.

In embodiments, the data structuring system 8106 may structure thevarious data collected by and/or on behalf of the enterprise. Inembodiments, the digital twin generation system 8108 generates theenterprise digital twins. As discussed, the digital twin generationsystem 8108 may receive a request for a particular type of digital twin(e.g., a CEO digital twin 8302 or a CTO digital twin 8310) and maydetermine the types of data needed to populate the digital twin based onthe configuration of the requested type of digital twin. In embodiments,the digital twin generation system 8108 may then generate the requesteddigital twin based on the various types of data (which may includestructured data structured by the data structuring system 8106). In someembodiments, the digital twin generation system 8108 may output thegenerated digital twin to a client application 8052, which may thendisplay the requested digital twins.

In embodiments, a CEO digital twin 8302 is a digital twin configured forthe CEO or analogous top-level decision maker of an enterprise. The CEOdigital twin 8302 may include high-level views of different statesand/or operations data of the enterprise, including real-time andhistorical representations of major assets, processes, divisions,performance metrics, the condition of different business units of theenterprise, and any other mission-critical information type. Inembodiments, the CEO digital twin 8302 may work in connection with theEMP 8000 to provide simulations, predictions, statistical summaries,decision-support based on analytics, machine learning, and/or other AIand learning-type processing of inputs (e.g., fiscal data, competitordata, product data, and the like). In embodiments, a CEO digital twin8302 may provide functionality including, but not limited to, managementof personnel, delegation of tasks, decisions, or tasks, coordinationwith the Board of Directors and/or strategic partners, risk management,policy management, oversight of budgets, resource allocation,investments, and other executive-related resources.

In embodiments, the types of data that may be populate a CEO digitaltwin 8302 may include, but are not limited to: macroeconomic data,microeconomic analytic data, forecast data, demand planning data,employment and salary data, analytic results of AI and/or machinelearning modeling (e.g., financial forecasting), prediction data,recommendation data, securities-relevant financial data (e.g., earnings,profitability), industry analyst data (e.g., Gartner quadrant),strategic competitive data (e.g., news and events regarding industrytrends and competitors), business performance metrics by business unitthat may be relevant to evaluating performance of the business units(e.g., P&L, head count, factory health, supply chain metrics, salesmetrics, R&D metrics, marketing metrics, and many others), Board packagedata, or some other type of data relevant to the operations of the CEOand/or executive department. In embodiments, the digital twin system8004 may obtain securities-relevant financial data from, for example,the enterprise's accounting software (e.g., via an API), publiclydisclosed financial statements, third-party reports, tax filings, andthe like. In embodiments, the digital twin system 8004 may obtainstrategic competitive data from public news sources, from publiclydisclosed financial reports, and the like. In embodiments, macroeconomicdata may be derived analytically from various financial and operationaldata collected by the EMP 8000. In embodiments, the business performancemetrics may be derived analytically, based at least in part on real timeoperations data, by the artificial intelligence services system 8010and/or provided from other users and/or their respective executivedigital twins. The CEO digital twin 8302 may be used to define real timeoperations data parameters of interest and to monitor, collect, analyze,and interpret real time operations data for conformance to and alignmentwith an organization's stated business objects, Board requirements,industry best practice, regulation, or some other criterion.

In embodiments, a CEO digital twin 8302 may include high-level views ofdifferent states of the enterprise, including real-time and historicalrepresentations of major assets, the condition of different businessunits of the enterprise, and any mission-critical information. The CEOdigital twin 8302 may initially depict the various states at a lowergranularity level. In embodiments, a user that is viewing the CEOdigital twin 8302 may select a state to drill down into the selectedstate and view the selected state at a higher level of granularity. Forexample, the CEO digital twin 8302 may initially depict a subset of thevarious states of the enterprise at a lower granularity level, includinga financial-department state (e.g., a visual indicator indicating anoverall financial health score of the enterprise). In response toselection, the CEO digital twin 8302 may provide data, analytics,summary, and/or reporting including, but not limited to, real-time,historical, aggregated, comparison, and/or forecasted financialinformation (e.g., real-time, historical, simulated, and/or forecastedrevenues, liabilities, and the like). In this way, the CEO digital twin8302 may initially present the user (e.g., the CEO) with a view ofvarious different aspects of the enterprise (e.g., different indicatorsto indicate different “health” levels of a respective business unit orpart of the enterprise) but may allow the user to select which aspectsrequire more of her attention. In response to such a selection, the CEOdigital twin 8302 may request a more granular view of the selectedstate(s) from the EMP 8000, which may return the requested states at themore granular level.

In embodiments, a CEO digital twin 8302 may include an executive-leveldigital twin of the executive department (e.g., C-suite, directors,Board members, and the like), which the user may use to identify,assign, instruct, oversee and review executive department personnel andthird-party personnel, departments, organizations and the like that areassociated with the activities of the executive of an organization,including the Board of Directors and the like that are involved in theoversight of the organization's management. In embodiments, theexecutive-level digital twin may include a definition of the variousroles, employees, and departments working under the CEO, the reportingstructure for each individual in the business unit and may be populatedwith the various names and/or other identifiers of the individualsfilling the respective roles. In embodiments, the CEO digital twin 8302may include a graphical user interface that provides the user theability to define/redefine personnel groupings, assign performancecriteria and metrics to business units, roles, and/or individuals,and/or assign/delegate tasks to business units, roles, and/orindividuals, and the like via the executive-level digital twin. Inembodiments, the executive-level digital twin may provide real-timeoperations data of the organization to continuously evaluate thepersonnel groupings' performance against the stored performancecriteria.

In embodiments, a CEO digital twin 8302 may be configured to interfacewith the collaboration suite 8006 to specify and provide a set ofcollaboration tools that may be leveraged by the executive departmentand associated parties. The collaboration tools may include videoconferencing tools, “in-twin” collaboration tools (e.g., where thecollaboration occurs to some extent within a common interface by whichthe digital twin entities are viewed and collaboration activities takeplace and/or where the components of the EMP that used to configure,operate or support the digital twin also govern collaboration arounddigital twin entities and workflows), whiteboard tools, agiledevelopment environment tools (such as features in Slack™ environments),presentation tools, word processing tools, spreadsheet tools, and thelike, as described herein. Collaboration and communication rules may beconfigured based at least in part on using the AI reporting tool, asdescribed herein. The collaboration tools may include collaborativecommunication (e.g., facilitating live conferencing where participantsare simultaneously presented with conference-related views of digitaltwin entities or workflows), asynchronous collaboration (such as whereactions on digital twin entities, comments, or the like are representedto different users who interact with the entities), version controlfeatures, and many others.

In embodiments, a CEO digital twin 8302 may be configured to provideresearch, track, and report on an executive department initiativeincluding, but not limited to, an overall strategic goal, policyimplementation, product roll-out, Board interaction, investment oracquisition, investor relations, public relations and press handling,budgeting, or some other type of executive initiative. The CEO digitaltwin 8302 may interact with and share such data and reporting with otherexecutive digital twins, including, but not limited to, a CFO digitaltwin, a COO digital twin, and the like. In embodiments, the CEO digitaltwin 8302 or an executive agent integrated with or within it (such asone trained to undertake expert executive actions as described elsewhereherein) may leverage intelligence services (e.g., data analytics,machine learning and A.I. processes) to analyze financial reports,projections, simulations, budgets, and related summaries to identify keydepartments, personnel, third-party or others that are, for example,listed in, or subject to, a project, initiative, budget line item andthe like, and who therefore may have an interest in such material. Suchmaterial pertaining to a given party may be abstracted and summarizedfor presentation, and formatted and presented automatically, or at thedirection of the CEO or other user, to the party that is the origin ofthe expense and/or subject of the material. For example, the CEO digitaltwin 8302 may assemble materials for the purposes of developingpresentations, speaking points, press releases, or some other materialfor the CEO or other executive personnel to use for public presentation.In an example, a CEO in anticipation of giving a conference presentationon the introduction of a new company product may use the CEO digitaltwin 8302 to specify and configure the identification, collection andassembly of operations data that is relevant to the upcomingpresentation, such as product data (e.g., units produced, unitsshipped), financial data (e.g., products sold, products reserved),graphic presentation information (e.g., product photos, maps of productdistribution, graphs of anticipated sales), forecasting data (e.g.,market growth expected), or some other type of data and assemble suchinformation in a presentation format, such as presentation slides, whitepaper template, speech talking points, press release, or some othersummary format that may form the basis of the presentation or bedistributed in conjunction with the presentation and/or its marketing.

In embodiments, a CEO digital twin 8302 may be configured to track andreport on stakeholder communications (e.g., reports, Board requests,investor requests) related to the executive department. The CEO digitaltwin 8302 may present, store, analyze, reconcile and/or report onexecutive activities related to parties with whom the executivedepartment is contracting, cooperating with, reporting to and so forth,such as key personnel, outside contractors, the press, the Board ofDirectors, or others.

In embodiments, the CEO digital twin 8302 may be configured to simulateone or more aspects of the enterprise. Such simulations may assist theuser (e.g., the CEO) in making executive level decisions. For example,simulations of a proposed executive initiative may be tested, forexample using the modeling, machine learning, and/or AI techniques, asdescribed herein, by simulating temporal effects on initiatives (e.g.,introduction of a new product), varying financial parameters (e.g.,potential investment levels), targeting parameters (e.g., geographic,demographic, or the like), and/or other suitable executive parameters.In embodiments, the digital twin simulation system 8116 may receive arequest to perform an executive simulation requested by the CEO digitaltwin 8302, where the request indicates one or more parameters that areto be varied in one or more enterprise digital twins. In response, thedigital twin simulation system 8116 may return the simulation results tothe CEO digital twin 8302, which in turn outputs the results to the uservia the client device display. In this way, the user may be providedwith various outcomes corresponding to different parameterconfigurations. For example, a user may request a set of simulations tobe run to test different supply chain strategies to see how thedifferent strategies affect the throughput of a manufacturing facilityand the overall impact on the profits and losses of the enterprise. Thedigital twin simulation system 8116 may perform the simulations byvarying the different supply chain strategies and may output thethroughputs and P&L forecasts for each respective supply chain strategy.In some embodiments, the user may select a parameter set based on thevarious outcomes, and iterate simulations based at least on the variedprior outcomes. Drawing from the previous example, the user may decideto select the supply chain strategy that maximizes P&L forecasts butdoes not adversely affect throughput of the manufacturing facility. Insome embodiments, an executive agent may be trained to recommend and/orselect a parameter set based on the respective outcomes associated witheach respective parameter set.

In embodiments, a CEO digital twin 8302 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to an executive strategy, executive planning, executiveactivities, and/or executive initiatives. For example, the CEO digitaltwin 8302 may be associated with a plurality of databases or otherrepositories of financial materials, summaries and reports andanalytics, including such materials, summaries and reports and analyticsrelated to prior executive activity (e.g., prior quarterly financialperformance, prior investments, prior strategic partners,co-developments, and the like), each of which may be further associatedwith financial and performance metrics pertaining to the campaign andwhich are also accessible to the CEO digital twin 8302.

In embodiments, a CEO digital twin 8302 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to financial reporting, ratings, rankings, financial trenddata, income data, or other data related to an executive'sresponsibilities. A CEO digital twin 8302 may link to, interact with,and be associated with external data sources, and able to upload,download, aggregate external data sources, including with the EMP'sinternal data, and analyze such data, as described herein. Dataanalysis, machine learning, AI processing, and other analysis may becoordinated between the CEO digital twin 8302 and an analytics teambased at least in part on using the artificial intelligence servicessystem 8010. This cooperation and interaction may include assisting withseeding executive-related data elements and domains in the enterprisedata store 8012 for use in modeling, machine learning, and AI processingto identify an optimal business strategy, or some otherexecutive-relating metric or aspect, as well as identification of theoptimal data measurement parameters on which to base judgement of anexecutive initiative's success. Examples of data sources 8020 that maybe connected to, associated with, and/or accessed from the CEO digitaltwin 8302 may include, but are not limited to, the sensor system 8022having sensors that sensor data from facilities (e.g., manufacturingfacilities, shipping and logistics facilities, transportationfacilities, agricultural facilities, resource extraction facilities,computing facilities, and many others) and/or other physical entities ofthe enterprise, the sales database 8024 that is updated with salesfigures in real time, the CRM system 8026, the content marketingplatform 8028, financial databases 8030, surveys 8032, org charts 8034,workflow management systems 8036, third-party data sources 8038,customer databases 8040 that store customer data, and/or third-partydata sources 8038 that store third-party data, edge devices 8042 thatreport data relating to physical assets (e.g., smartmachinery/manufacturing equipment, sensor kits, autonomous vehicles ofthe enterprise, wearable devices, and the like), enterprise resourcemanagement systems 8044, HR systems 8046, content management systems8016, and the like). In embodiments, the digital twin system 8004abstracts the different views (or states) within the digital twin to theappropriate granularity. For instance, the digital twin system 8004 mayhave access to all the sensor data collected on behalf of the enterpriseas well as access to real-time sensor data streams. Typically, such datais far too granular for an executive such as a CEO, and sensor datareadings are often of little importance to the CEO unless associatedwith a mission critical state or operation. In this example, however, ifthe sensor readings from a particular physical asset (e.g., a criticalpiece of manufacturing equipment) are indicative of a potentiallycritical situation (e.g., failure state, dangerous condition, or thelike), then the analytics that indicate the potentially criticalsituation may become very important to the CEO. Thus, the digital twinsystem 8004 may, when building the appropriate perspective for the CEO,include a state indicator of the physical asset in the CEO digital twin.In this way, the CEO can drill down into the state indicator of thephysical asset to view the potentially critical situation at a greatergranularity (e.g., the machinery and an analysis of the sensor data usedto identify the situation).

In embodiments, a CEO digital twin 8302 may be configured to monitor anorganization's performance based at least in part on real timeoperations data and the use of the monitoring agent of the clientapplication 8052, as described herein, that is associated with the CEOdigital twin 8302. The monitoring agent may report on such activities tothe EMP 8000 for presentation in a user interface that is associatedwith the CEO digital twin 8302. In response, the EMP 8000 may train anexecutive agent (which may include one or more machine-learned models)to handle and process such notifications when they next arrive, andescalate and/or alert the CEO when such notifications are of an urgentnature, such as an announcement of an acquisition by a competitor, areport indicating an under-performing business unit, a high-profilepress article, a radical change in the stock market (for the CEO'scompany, a cohort member, or the market as a whole), a downgrade inrating by an industry analyst, an external event likely to disruptoperations (such as a natural disaster or epidemic) or some otherimportant event. In embodiments, the CEO digital twin 8302 may generateperformance alerts based on real time operations data, performancetrends, and the like. This may allow a CEO to optimize initiatives inreal-time without having to manually request such real-time data; theCEO digital twin 8302 may automatically present such information andrelated/necessary alerts as configured by the organization, CEO, or someother interested party.

In embodiments, a CEO digital twin 8302 may be configured to report onthe performance of the executive department, personnel of the executivedepartment, executive activities, executive content, executiveplatforms, executive partners, or some other aspect of management withina CEO's responsibilities. Reporting may be to the CEO, the executivedepartment, to other executives of an organization (e.g., the COO), orto outside third parties (e.g., partners, press releases, and the like).As described herein, reporting may include stakeholder summaries,minutes of meetings, presentations, sales data, customer data, financialperformance metrics, personnel metrics, data regarding resource usage,industry summaries (e.g., summaries of merger and acquisition activityin an industry segment), or some other type of reporting data. Reportingand the content of reporting may be shared by the CEO digital twin 8302with other executive digital twins. The reporting functionality of theCEO digital twin 8302 may also be used for populating new or presetreporting formats, and the like. Templets of common reporting formatsmay be stored and associated with the CEO digital twin 8302 to automatethe presentation of data and analytics according to pre-defined formats,styles and system requirements. In embodiments, an executive agenttrained by the user may be trained to surface the most important reportsto the user. For example, if the user (e.g., the CEO) consistently viewsand follows up on sales data reports but routinely skips over reportsrelating to the manufacturing KPIs, the executive agent mayautomatically surface sales data reports to the user and mayautomatically delegate manufacturing KPIs to another executive digitaltwin (e.g., the COO digital twin).

In embodiments, a CEO digital twin 8302 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to competitors of a CEO's organization, or namedentities of interest. In embodiments, such data may be collected by theEMP 8000 via data aggregation, spidering, web-scraping, or othertechniques to search and collect competitor information from sourcesincluding, but not limited to, information on investment and/oracquisitions, press releases, SEC or other financial reports, or someother publicly available data. For example, a user wishing to monitor acertain competitor may request that the CEO digital twin 8302 providematerials relating to the certain competitor. In response, the EMP 8000may identify a set of data sources that are either publicly available orto which the enterprise of the CEO has access (e.g., internal datasources, licensed third-party data, or the like). The EMP 8000 mayconfigure a cohort digital twin 8320 based on the types ofdata/analysis/services the user requests and the identified set of datasources. The EMP 8000 may then serve the cohort digital twin 8320associated with the requested party (e.g., competitor) to the CEOdigital twin 8302.

In embodiments, a CEO digital twin 8302 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to regulatory activity, such as governmentregulations, industry best practices or some other requirement orstandard. For example, the CEO digital twin 8302 may be in communicationwith another enterprise digital twin, such as a General Counsel digitaltwin 8314, through which the legal team can keep the CEO apprised of newregulation or regulation changes as they occur.

In embodiments, the client application 8052 that executes the CEOdigital twin 8302 may be configured with an executive agent 8364 that istrained on the CEO's actions (which may be indicative of behaviors,and/or preferences). In embodiments, the executive agent 8364 may recordthe features relating to the actions (e.g., the circumstances relatingto the user's action) to the expert agent system 8008. For example, theexecutive agent 8364 may record each time the user delegates a task to asubordinate (which is the action) as well as the features surroundingthe delegation of the task (e.g., an event that caused the user todelegate the task, the type of task that was delegated, the role towhich the task was delegated, instructions provided by the user with thedelegation, and the like). The executive agent 8364 may report theactions and features to the expert agent system 8008 and the expertagent system 8008 may train the executive agent 8364 on the manner bywhich the executive agent 8364 can delegate or recommend delegation oftasks in the future. Once trained, the executive agent 8364 mayautomatically perform actions and/or recommend actions to the user.Furthermore, in embodiments, the executive agent 8364 may recordoutcomes related to the performed/recommended actions, thereby creatinga feedback loop with the expert agent system 8008.

References to features and functions of the EMP and digital twins inthis example of a CEO digital twin 8302 should be understood to apply toother digital twins, and their respective projects and workflows, exceptwhere context indicates otherwise.

In embodiments, a Chief Financial officer (CFO) digital twin 8304 may bea digital twin configured for a CFO of an enterprise, or an analogousexecutive tasked with overseeing the finance-related tasks of theenterprise. A CFO digital twin 8304 may provide data, analytics,summary, and/or reporting including, but not limited to, real-time,historical, aggregated, comparison, and/or forecasted financialinformation (e.g., real-time, historical, simulated, and/or forecastedsales figures, expenditures, revenues, liabilities, and the like). Inembodiments, the CFO digital twin may work in connection with the EMP8000 to provide simulations, predictions, statistical summaries,decision support based on analytics, machine learning, and/or other AIand learning-type processing of inputs (e.g., accounting data, salesdata, sensor data and the like).

In embodiments, a CFO digital twin 8304 may provide features andfunctionality including, but not limited to, management of financialpersonnel, partners and outside consultants and contractors (e.g.,accounting firms, auditors and the like), oversight of budgets,procurement, expenditures, receivables, and other finance-relatedresources, compliance, oversight of sales and sales staff anddepartments' financial performance, management of contracting,management of internal policies (e.g., policies related to expendituresand reporting), tax law, finance-related privacy law (e.g., pertainingto credit agency data), reporting, compliance, and regulatory analysis.

In embodiments, the types of data that may populate a CFO digital twinmay include, but are not limited to, financial performance metrics bybusiness unit, by product, by geography, by factory, by storelocation(s), by asset class, earnings, cash, balance sheet data, cashflow, profitability, resource utilization, audit data, general ledgerdata, asset performance data, securities and commodities data, insuranceand risk management data, asset aging and depreciation data, assetallocation data, macroeconomic data, microeconomic analytic data, taxdata, pricing data, competitive product and pricing data, forecast data,demand planning data, employment and salary data, analytic results of AIand/or machine learning modeling (e.g., financial forecasting),prediction data, recommendation data, or some other type of datarelevant to the operations of the CFO and/or finance department. Inembodiments, “datum,” “data,” “dataset,” “datastore,” “data warehouse,”and/or “database,” as used herein, may refer to information that isstored in a numeric or statistical format, including summaries, inputsor outputs in statistical or scientific notation, and also includesinformation that is stored in natural language format (e.g., textexcerpts from reports, press releases, statutes and the like),information in a graphic format (e.g., financial performance graphs),information in audio and/or audio-visual format (e.g., recorded audiofrom conference calls or video from presentations, including naturallanguage transcript summaries of audio and/or audio-visual formattedinformation), or some other type of information.

In embodiments, a CFO digital twin 8304 may depict a finance departmenttwin of the finance department, which the user may use to identify,assign, instruct, oversee and review finance department personnel andthird-party personnel that are associated with the finance activities ofan organization, including third-party partners and other outsidecontractors, such as accounting firms, tax lawyers and the like that areinvolved in the organization's finance endeavors. Examples of suchorganization personnel include, but are not limited to, financedepartment staff, sales analysts, statisticians, data scientists,executive personnel, human resources staff, Board Members, advisors, orsome other type of organization personnel relevant to the functioning ofa finance department. Examples of a finance department's third-partypersonnel include, but are not limited to, lawyers, accountants,management consultants, social media platform personnel, financepartners, consultants, contractors, financial firm staff, auditors, orsome other type of third-party personnel.

In embodiments, the CFO digital twin 8304 may include a definition ofthe various roles/employees working under the CFO, the reportingstructure, and associated permissions, for each individual in thebusiness unit, and may be populated with the various names and/or otheridentifiers of the individuals filling the respective roles. Inembodiments, a user (e.g., the CFO of an enterprise) may use the CFOdigital twin 8304 to adjust the reporting structure within the financedepartment and/or to grant permissions to one or more individuals withinthe department.

In embodiments, a CFO digital twin 8304 may be configured to interfacewith the collaboration suite 8006 to specify and provide a set ofcollaboration tools that may be leveraged by the finance department andassociated parties. The collaboration tools may include videoconferencing tools, “in-twin” collaboration tools, whiteboard tools,presentation tools, word processing tools, spreadsheet tools, and thelike, as described herein. Collaboration and communication rules may beconfigured based at least in part on using the AI reporting tool, asdescribed herein.

In embodiments, a CFO digital twin 8304 may be configured to research,create, track and report on a finance department initiative including,but not limited to, an overall department budget, a budget for a singleor group of finance initiatives, an audit, a third-party vendoractivity, or some other type of expense or budget. In embodiments, theCFO digital twin 8304 may interact with and share such expense or budgetdata and reporting with other enterprise twins, as described herein,including, but not limited to, a digital twin related to accountspayable, executive staff such as the CEO (e.g., CEO digital twin) or COO(e.g., the COO digital twin), or other suitable enterprise digitaltwins. In embodiments, the CFO digital twin 8304 may leverage one ormore intelligence services of the EMP 8000 based at least in part on thedata analytics, machine learning and A.I. processes, as describedherein, to provide financial reports, projections, simulations, budgetsand related summaries. In some of these embodiments, the CFO digitaltwin 8304 may use the intelligence services to identify key departments,personnel, third-party or others that are, for example, listed in, orsubject to, the budget line item and who therefore may have an interestin such material. Budget material pertaining to a given party may beabstracted and summarized for presentation independent from the entiretyof the budget, and formatted and presented automatically, or at thedirection of the CFO or other user, to the party that is the origin ofthe expense and/or subject of the budget item.

In some embodiments, a CFO digital twin 8304 may be configured to trackand report on inbound and outbound billing (i.e., accounts receivableand payable) related to the finance department and/or organization. Inembodiments, the CFO digital twin 8304 may include a billing digitaltwin that identifies the billing department, personnel, processes andsystems associated with the billing workflows of the enterprise. Inthese embodiments, the billing digital twin may interact present, store,analyze, reconcile and/or report on billing activities related toparties with whom the finance department is interacting. In someembodiments, the user of the CFO digital twin 8304 may approve bills,issue bills, drill down into a set of bills, initiate investigations ofbills or the like via the GUI if the CFO digital twin 8304.

In embodiments, a CFO digital twin 8304 may be configured to provide auser (e.g., a CFO or other finance department executive) withinformation that is unique to the CFO digital twin 8304 and thus canprovide insights and perspectives on financial performance that areunique to the CFO digital twin 8304. For example, in supply chainplanning, demand forecasting, operational planning and other of theCFO's activities, traditional data sources, models and projections maybe “siloed” in ways, meaning they may be quantitatively robust within aparticular domain, but that domain may be constrained by factorsincluding, but not limited to, the origins of the data, the formatwithin which the data is recorded, the statistical weights used increating or transforming the data that is available, or some otherconstraint. In embodiments, the EMP 8000 in connection with the CFOdigital twin 8304 may create and derive new financial metrics andanalytics including, but not limited to, functionalities such as nativedata and model creation, and data and model combinations andaggregations based at least in-part on the real time operations of anorganization. Native data and model creation, such as specifying thedata to be collected, the format within which to collect and store thedata, the data transformations to model, and so forth gives one theability to craft, combine, aggregate, modify, transform, and/or weightthe native data (including in combination with other third-party data)in manners that are appropriately mathematically tuned to the modeling,analysis, machine learning, and/or AI techniques that are performed bythe EMP 8000 and CFO digital twin 8304, rather than being reliant ondata and/or model presets. Similarly, in the analytic context of theCFO's operations and the function of the EMP and CFO digital twin 8304,native data and model creation and structuring by the EMP and CFOdigital twin 8304 enables analytics, machine learning, AI operations andthe like, yielding new analytic results and insights, based at least inpart on the real time operations of an organization, because the EMP andCFO digital twin 8304 has enabled the CFO to move further up infinancial data creation and modeling operations to assert greatercreative control over the types of data and other input material to beused in developing analytic insights that may be created and reportedfor the purpose of improving performance including, but not limited to,product margins (e.g., gross, contribution, net and the like), productfeatures, upsell opportunities or some other performance metric.

In embodiments, the CFO digital twin 8304 may be configured to simulatefinance-related activities on behalf of a user. In these embodiments,the user may identify one or more parameters that can be varied duringfor a simulation including, but not limited to, financial and/or budgetparameters, pricing and sales goal settings, process designs, andmaintenance/infrastructure upgrades, internal controls design, producttesting frequencies/types, manufacturing down-times, flexible workforceplanning, and the like. In these embodiments, the digital twinsimulation system 8116 may receive a request to perform the simulationrequested by the CFO digital twin 8304, where the request indicatesfeatures and the parameters, including financial parameters, that are tobe varied. In response, the digital twin simulation system 8116 mayreturn the simulation results to the CFO digital twin 8304, which inturn outputs the results to the user via the client device display. Inthis way, the user is provided with various outcomes corresponding todifferent parameter configurations. In some embodiments, the user mayselect a parameter set based on the various outcomes. In someembodiments, an executive agent trained by the user may select theparameter sets based on the various outcomes. The simulations, analyticsand/or modeling performed by the CFO digital twin 8304 may be used tomitigate risk for IPO, M&A, equity and debt offerings, or some othertype of transaction. The simulations, analytics and/or modelingperformed by the CFO digital twin 8304 may be used to create andstructure sales incentives, including commissions and otherperformance-based compensation. The simulations, analytics and/ormodeling performed by the CFO digital twin 8304 may be used to evaluateinsurance offerings and other information related to businessinterruption preparedness. The simulations, analytics and/or modelingperformed by the CFO digital twin 8304 may be used to analyze loancovenant monitoring and projections. The CFO equipped with digital twin8304 will be better able to adapt quickly to change by predictingheadwinds, forecasting operational performance, and making informeddecisions across departments while mitigating risk.

In embodiments, a CFO digital twin 8304 may be configured to manageoperational planning, based at least in part by leveraging predictiveanalytics for sales planning, and supply chain management in order toincrease company efficacy while optimizing operating expenses.

In embodiments, a CFO digital twin 8304 may be configured to accessinsights across environmental resource management (ERM) solutions forrisk oversight that includes, but is not limited to, internal controlsdesign, testing, certification, and reporting while directing listedactions into a repository. In embodiments, a CFO digital twin 8304 maybe configured to streamline governance, risk management, and complianceprocesses in order to connect risk and compliance across theorganization and manage complex audit fieldwork and work papers.

In embodiments, a CFO digital twin 8304 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to a financial strategy, plan, activity or initiative. Forexample, the CFO digital twin 8304 may be associated with a plurality ofdatabases or other repositories of financial materials, summaries andreports and analytics, including such materials, summaries and reportsand analytics related to prior financial activity (e.g., prior quarterlyfinancial performance), each of which may be further associated withthird-party financial or economic data.

In embodiments, a CFO digital twin 8304 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to financial reporting, ratings, rankings, financial trenddata, income data, or other finance department-related data. A CFOdigital twin 8304 may link to, interact with, and be associated withexternal data sources, and able to upload, download, aggregate externaldata sources, including with the EMP's internal data, and analyze suchdata. Data analytics, machine learning, AI processing, and otherdata-driven processes may be coordinated between the CFO digital twin8304 and an analytics team based at least in-part on insights derived bythe artificial intelligence services system 8010. This cooperation andinteraction may include assisting with seeding finance-related dataelements and domains in the enterprise data store 8012 for use inmodeling, machine learning, and AI processing to identify the optimalfinancial strategy, or some other finance-related metric or aspect, aswell as identification of the optimal data measurement parameters onwhich to base judgement of a finance endeavor's success. Examples ofdata sources 8020 that may be connected to, associated with, and/oraccessed from the CFO digital twin 8304 may include, but are not limitedto, the sensor system 8022, the sales database 8024 that is updated withsales figures in real time, the CRM system 8026, news websites 8048, thefinancial database 8030 that tracks costs of the business, an org chart8034, a workflow management system 8036, customer databases 1S40 thatstore customer data, and/or third-party data sources 8038 that storethird-party data.

In embodiments, a CFO digital twin 8304 may aggregate data sources andtypes, creating new data types, summaries and reports that are notavailable elsewhere. This may reduce reliance upon the need of multiplethird-party providers and current solutions. This may, among otherbenefits and improvements, reduce expenses associated with acquiringdata needed for sound financial decision making.

In embodiments, a CFO digital twin 8304 may be configured to monitor auser's performance of finance-related tasks via a monitoring function ofan agent of the client application 8052 executing the CFO digital twin8304. In embodiments, the monitoring function of the executive agent mayreport on certain activities to the EMP 8000 that are undertaken by theuser when interfacing with the CFO digital twin 8304. In response, theEMP 8000 may train the executive agent (which may include one or moremachine-learned models) to handle and process such finance-related taskswhen they next arrive. For example, the monitoring function may monitorwhen the user (e.g., the CFO) escalates a state of the CFO digital twin8304 to the CEO and/or when the user delegates a task to a subordinatevia the CFO digital twin 8304. Each time such escalations and/ordelegation events occur and/or when the user (e.g., the CFO or otherfinance executive) responds to an alert or other notifications of anurgent nature and may report and may report the actions taken by theuser in response to each respective account to the EMP 8000. Inresponse, the expert agent system 8008 may train an executive agent 8364based on the reported actions, which in turn may be leveraged by the CFOdigital twin to respond to certain later occurring events on which theexecutive agent 8364 was trained on (e.g., analytics showing poorfinancial performance or finance activity (e.g., a new investment). Forexample, an executive agent 8364 trained with respect to a CFO digitaltwin 8304 may automatically issue financial performance alerts tocertain employees based on performance trends of one or more businessunits. In another example, the executive agent 8304 may automaticallyescalate a notification to the CEO (which may be depicted in the CEOdigital twin 8302) when certain metrics indicate a poor financialforecast. In embodiments, the executive agent 8364 in connection withthe CFO digital twin 8304 may allow a CFO to optimize initiatives inreal-time without having to manually request such real-time financialperformance data. In some embodiments, the CFO digital twin 8304 mayautomatically present such information and related/necessary alerts asconfigured by the configuring user, the CFO, or some other user havingsuch permissions.

In embodiments, an executive agent 8364 trained in connection with a CFOdigital twin 8304 may be configured to report on the performance of thefinance department, personnel of the finance department, financeactivities, finance content, finance platforms, finance partners, orsome other aspect of management within a CFO's responsibilities.Reporting may be to the CEO, the Board of Directors, other executives ofan organization (e.g., the COO), or to outside third parties (e.g.,partners, press releases, and the like). The reporting functionality ofthe CFO digital twin 8304 may also be used for populating required datafor formal reporting requirements such as shareholder statements, annualreports, SEC filings, and the like. Templets of common reporting formatsmay be stored and associated with the CFO digital twin 8304 to automatethe presentation of data and analytics according to pre-defined formats,styles and system requirements.

In embodiments, a CFO digital twin 8304 in combination with the EMP 8000may be configured to monitor, store, aggregate, merge, analyze, prepare,report and distribute material relating to competitors of a CFO'sorganization, or named entities of interest. In embodiments, such datamay be collected by the EMP 8000 via data aggregation, spidering,web-scraping, or other techniques to search and collect competitorinformation from sources including, but not limited to, press releases,SEC or other financial reports, mergers and acquisitions activity, orsome other publicly available data.

In embodiments, a CFO digital twin 8304 in combination with the EMP 8000may be configured to monitor, store, aggregate, merge, analyze, prepare,report and distribute material relating to regulatory activity, such asgovernment regulations, industry best practices or some otherrequirement or standard. For example, the CFO digital twin 8304 may bein communication with another enterprise digital twin, such as a GeneralCounsel digital twin 8314, through which the legal team can keep the CFOapprised of new regulations or regulation changes as they occur.

In embodiments, the client application 8052 that executes the CFOdigital twin 8304 may be configured with an executive agent that reportsa CFO's behaviors and preferences (or other finance personnel'sbehaviors and preferences) to the expert agent system 8008, as describedherein, and the expert agent system 8008 may train the executive agenton how the CFO or other finance personnel respond to certain situationsand adjust its operation based at least in part on the data collection,analysis, machine learning and A.I. techniques, as described herein. Theforegoing examples are optional examples and are not intended to limitthe scope of the disclosure.

References to features and functions of the EMP and digital twins inthis example of a finance department and a CFO digital twin 8304 shouldbe understood to apply to other departments and digital twins, and theirrespective projects and workflows, except where context indicatesotherwise.

In embodiments, a Chief Operating officer (COO) digital twin 8306 may bea digital twin configured for a COO of an enterprise, or an analogousexecutive tasked with overseeing the operations tasks of the enterprise.A COO digital twin 8306 may provide functionality including, but notlimited to, management of personnel and partners, oversight of variousdepartments (e.g., oversight over marketing department, HR department,sales department, and the like), project management, implementationand/or rollouts of business processes and workflows, budgeting,reporting, and many other operations-related tasks.

In embodiments, a COO digital twin 8306 may provide data, analytics,summary, and/or reporting including, but not limited to, real-time,historical, aggregated, comparison, and/or forecasted financialinformation (e.g., sales, expenditures, revenues, liabilities,profitability, cash flow and the like), mergers and acquisitionsinformation, systems data, reporting and controls data, or some otheroperations related information. In embodiments, the COO digital twin8306 may work in connection with the EMP 8000 to provide simulations,predictions, statistical summaries, decision support based on analytics,machine learning, and/or other AI and learning-type processing of inputs(e.g., equipment data, sensor data and the like), for example thoserelated to the development, communication and implementation ofeffective growth strategies and processes for an organization.

In embodiments, the types of data that may populate a COO digital twinmay include, but are not limited to, operations data, key performanceindicators (KPIs) for factories/plants, business units,assets/equipment; uptime/downtime, safety data, risk management data,supply chain/component availability data, demand plan data, logisticsdata, workflow data, financial performance metrics by business unit, byproduct, by geography, by factory, by store location(s), by asset class,earnings, resource utilization; audit data, asset performance data,asset aging and depreciation data, asset allocation data, or some othertype of operations-relevant data or information.

In embodiments, a COO digital twin 8306 may depict a twin of theoperations department, which the user may use to identify, assign,instruct, oversee and review operations department personnel andthird-party personnel that are associated with the design,implementation and evaluation of operational processes, internalinfrastructures, reporting systems, company policies, and the like.

In embodiments, the COO digital twin 8306 may include a definition ofthe various roles/employees working under the COO, the reportingstructure, and associated permissions, for each individual in thebusiness unit, and may be populated with the various names and/or otheridentifiers of the individuals filling the respective roles.

In embodiments, a COO digital twin 8306 may be configured to interfacewith the collaboration suite 8006 to specify and provide a set ofcollaboration tools that may be leveraged by the operations departmentand associated parties. The collaboration tools may include videoconferencing tools, “in-twin” collaboration tools, whiteboard tools,presentation tools, word processing tools, spreadsheet tools, and thelike, as described herein. Collaboration and communication rules may beconfigured based at least in part on using the AI reporting tool, asdescribed herein.

In some of these embodiments, the COO digital twin 8306 may beconfigured to simulate operations activities, such as a proposed newoperational plan, process or program. In these embodiments, the digitaltwin simulation system 8116 may receive a request to perform thesimulation requested by the COO digital twin 8306, where the requestindicates features and the parameters of the operational plan or otheractivity that is proposed for implementation, the associated variablesfor which may be altered or varied to produce differing simulationenvironments. In response, the digital twin simulation system 8116 mayreturn the simulation results to the COO digital twin 8306, which inturn outputs the results to the user via the client device display. Inthis way, the user is provided with various outcomes corresponding todifferent operational parameter configurations. In embodiments, anexecutive agent trained by the user may select the parameter sets basedon the various outcomes.

In embodiments, a COO digital twin 8306 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to an operations strategy, plan, activity or initiative. Forexample, the COO digital twin 8306 may be associated with a plurality ofdatabases or other repositories of operational data, summaries andreports and analytics, including such materials, summaries and reportsand analytics related to prior operations activity, each of which may befurther associated with financial and performance metrics pertaining tothe activity and which are also accessible to the COO digital twin 8306.

In embodiments, a COO digital twin 8306 may be configured to monitoroperational performance, including in real time, based at least in parton use of the monitoring agent of the client application 8052, asdescribed herein, that is associated with the COO digital twin 8306. Themonitoring agent may report on such activities to the EMP 8000 forpresentation in a user interface that is associated with the COO digitaltwin 8306. In response, the EMP 8000 may train an executive agent (whichmay include one or more machine-learned models) to handle and processsuch notifications when they next arrive and escalate and/or alert theCOO when such notifications are of an urgent nature.

In embodiments, a COO digital twin 8306 may be configured to report onthe performance of the operations department, personnel of theoperations department, operations activities, operations content,operations platforms, operations partners, or some other aspect ofmanagement within a COO's responsibilities.

In embodiments, the EMP 100 trains and deploys executive agents onbehalf of enterprise users. In embodiments, an executive agent is anAI-based software system that performs tasks on behalf of and/orsuggests actions to a respective executive user. In embodiments, the EMP100 receives data from various data sources associated with a particularentity or workflow and learns the workflows performed by the particularuser based on the data and the surrounding circumstances or context. Forexample, the user may be a COO that is presented a COO digital twin8306. Among the responsibilities of the COO may be schedulingmaintenance and replacement of equipment in a manufacturing, warehouse,or other operational facility. The states depicted in the COO digitaltwin 8306 may include depictions of the condition of different pieces ofequipment within the operational facility. In this example, the COO mayschedule maintenance via the digital twin when a piece of equipment isdetermined to be in a first condition (e.g., a deteriorating condition)and may issue a request to the COO via the COO digital twin 8306 toreplace the piece of equipment when the equipment is determined to be ina second condition (e.g., a critical condition). The executive agent maylearn the COO's tendencies based on the COO's previous interaction withthe COO digital twin 8306. Once trained, the executive agent mayautomatically request replacements from the COO when a particular pieceof equipment is determined to be in the second condition and mayautomatically schedule maintenance if the piece of equipment is in thefirst condition.

In embodiments, the client application 8052 that executes the COOdigital twin 8306 may be configured with an executive agent that reportsa COO's behaviors and preferences (or other operations personnel'sbehaviors and preferences) to the executive agent system 8008, asdescribed herein, and the executive agent system 8008 may train theexecutive agent on how the COO or other executive personnel respond tocertain situations and adjust its operation based at least in part onthe data collection, analysis, machine learning and A.I. techniques, asdescribed herein. The foregoing examples are optional examples and arenot intended to limit the scope of the disclosure.

References to features and functions of the EMP and digital twins inthis example of an operations department and a COO digital twin 8306should be understood to apply to other departments and digital twins,and their respective projects and workflows, except where contextindicates otherwise.

In embodiments, a Chief Marketing officer (CMO) digital twin 8308 may bea digital twin configured for a CMO of an enterprise, or an analogousexecutive tasked with overseeing the marketing tasks of the enterprise.A CMO digital twin 8308 may provide functionality including, but notlimited to, management of personnel and partners, development andoversight of marketing budgets and resources, management of marketingand advertising platforms, development and management of marketingcontent, strategies and campaigns, reporting, competitor analysis,regulatory analysis, and management of data privacy and security.

In embodiments, the types of data that may populate and/or be utilizedby a CMO digital twin 8308 may include, but are not limited to,macroeconomic data; market pricing data; competitive product and pricingdata; microeconomic analytic data; forecast data; demand planning data;competitive matrix data; product roadmap; product capability data;consumer behavior data; consumer profile data; collaborative filteringdata; analytic results of AI and/or machine learning modeling; channeldata; demographic data; geographic data; prediction data; recommendationdata, or some other type of data relevant to the operations of the CMOand/or marketing department.

In embodiments, an executive digital twin, such as a CMO digital twin8308 or other executive digital twin may depict a twin of a department,such as the marketing department or other department, which the user mayuse to identify, assign, instruct, oversee and review departmentpersonnel and third-party personnel that are associated with theactivities of a particular department of an organization, includingthird-party partners and other outside associates involved in theorganization's related endeavors. Examples of such organizationpersonnel include, but are not limited to, an organization's marketingstaff, sales staff, finance staff, product design personnel, engineers,analysts, statisticians, data scientists, advertising staff, executivepersonnel, human resources staff, Board Members, advisors, or some othertype of organization personnel. Examples of an organization'sthird-party personnel include, but are not limited to, advertising firmstaff, ad exchange staff, outside creative or content developers, socialmedia platform personnel, co-marketing partners, consultants,contractors, financial firm staff, auditors, or some other type ofthird-party personnel. In embodiments, the departmental twin (in thisexample a marketing department twin) may include a definition of thevarious roles/employees working under the executive (e.g., CMO), thereporting structure, and associated permissions, for each individual inthe business unit, and may be populated with the various the namesand/or other identifiers of the individuals filling the respectiveroles. In embodiments, the department twin (e.g., marketing departmenttwin) may include subsections that are specific to an activity orinitiative, such as a marketing or advertising campaign. In this way,the executive (e.g., a CMO) may easily identify the personnel andthird-party providers that are involved in the initiative and/or assignindividuals and/or third parties to the initiative. A user may defineone or more restrictions, permissions, and/or access rights of theindividuals indicated in the business unit (e.g., using the enterpriseconfiguration system 8002), as described herein, such that therestrictions, permissions, and/or access rights can be controlled by theCMO (or analogous user). In embodiments, the permissions to define suchrestrictions and/or rights may be, for example, defined in theorganizational digital twin that lists the user as having a role thatpermits implementing permissions, restrictions, and/or access rights toroles/individuals In embodiments, a personnel restriction or rightassociated with a role/individual may be specific to a project, such asa marketing or advertising campaign, and may define one or more types ofdata that a particular user or group of users is allowed, or notallowed, to access (either directly or in a digital twin). For example,a first marketing campaign twin may allow a marketing departmentemployee to review the first marketing budget for a first marketingcampaign and approve marketing expenditures for the first marketingcampaign up to $10,000, but a second marketing campaign twin maydisallow the same employee from any budgetary review or expenditures.Similar approaches can be used by projects of various types across anorganization and its departments, such as product development projects,logistics projects, corporate development projects, service projects,and many others. In embodiments, a breach, or attempted breach, of arestriction, permission or access right may invoke a notice, alert,warning or some other action to an individual notifying them of thebreach or attempted breach. In an example such a notice, alert, orwarning may be sent to an individual that is identified based at leastin part on the individual's position in the org chart relative to theperson breaching or attempting to breach a restriction, permission oraccess right. In another example, such a notice, alert, or warning maybe sent to an individual that is not identified in a departmental orgchart and/or specific project or campaign, but rather may be sent to anindividual that is identified based at least in part on a rule that isdefined in the organizational twin of the entire enterprise. Forexample, a rule stored within an organizational digital twin of theentity may specify that an alert must be sent to an Information SecurityDepartment staff member, or some other staff member, upon an attemptedlogin to a forbidden file, or other, system. Other rules may be relatedto geographic, temporal, or other types of restrictions, as describedherein. In embodiments, an alert may be an email, phone call, text, orsome other communication type.

In embodiments, a CMO digital twin 8308 may be configured to oversee andmanage personnel and human resources issues and activities related tothe marketing department. For example, a marketing department twin maymap each individual within the marketing department to her respectivemarketing department. Using the CMO digital twin 8308, the user may beable to select a department to see greater detail on the functioning ofthe department. Alternatively, this step may be automatically performedby the CMO digital twin 8308, requiring no action from the user (e.g.,the CMO) (e.g., via an executive agent trained by the user). Forexample, the greater detail might include the number of vacanciescurrently associated with the department and the duration that each ofthe open positions has remained unfilled, estimated salary dataassociated with the open positions, and the like. The user may be ableto also select to see more information on the budget associated with agiven department, such as a department with a personnel vacancy, inorder to see if there is currently available budget to cover a new hirefor the department. Alternatively, this step may be automaticallyperformed by the CMO digital twin 8308, requiring no action from theuser. Continuing the example, if there is budget to cover a new hire,the CMO digital twin 8308 may provide a link or other opportunity forthe user to initiate a communication with human resources or some otherdepartment personnel to begin the process of posting a job listing.Alternatively, this step may be automatically performed by the CMOdigital twin 8308 (e.g., via an executive agent executing on behalf ofthe user), requiring no action from the user. This communication may bedrawn from a repository of form emails, letters or other communicationsso that the user need not compose the communication, but rather onlysignal within the CMO digital twin 8308 that such communication shouldbe sent. Similarly, based on the communication type (e.g., “initiate anew marketing job posting”) the user may not need to select thereceiving party, whom may be stored in the EMP as the appropriaterecipient based at least in part on a rule associated with thecommunication type. Continuing the example further, alternatively, ifthere is not budget available to cover a new hire, a second type ofcommunication may be invoked by the CMO digital twin 8308, for example,an email, calendar invitation to reserve a meeting, or some other typeof communication may be selected to be sent to the CFO, or otherfinancial personnel, to request a meeting to discuss the marketingdepartment's budget or initiate some other activity. Following thisexample, if and when the new hires are approved, the CMO digital twinmay allow the user to delegate the hiring task to a subordinate orherself. In the event the user is assigned the hire the new employee,the CMO digital twin 8308 may provide materials regarding candidates(e.g., resume, referrals, interview notes from interviewers, or thelike) and the user may select one or more candidates to furtherconsider, interview, or hire.

In an example, a user may be able to select a sub-department within themarketing department to view the performance of the sub-department ingreater detail. For example, the greater detail might include the numberof types of training sessions, tutorials, events, conferences, and thelike that personnel in the selected marketing department have received.The user may be able to compare such training and event attendancelevels with a specified target criterion that is stored in EMP, or thatis associated with the EMP. This may result in the CMO digital twin 8308reporting to the CMO a listing of personnel in her department whosetraining and/or event attendance fails to meet the target criterion.This listing may be prioritized by the CMO digital twin 8308 tohighlight those staff members most in need of further training. The usermay be able to also select to see more information on the budgetassociated with a given department, such as a department with staff whodo not have adequate training according to the target criterion, inorder to see if there is currently available budget to cover additionaltraining for the department. If there is budget to cover additionaltraining, the CMO digital twin 8308 may provide, for example, a link orother opportunity for the user to initiate a communication to a staffmember in need of training to alert them that they must scheduletraining and/or attendance at an event within a timeframe. Thiscommunication may be drawn from a repository of form emails, letters orother communications so that the user need not compose thecommunication, but rather only signal within the CMO digital twin 8308that such communication should be sent. Continuing the example further,a second type of communication may be invoked by the CMO digital twin8308, for example, a request for information, training registration, orsome other type of communication may be selected to be sent to athird-party training vendor that is used by the marketing department, aconference event registration, or other training or event entity, torequest scheduling training and/or event registration, or some otheractivity. Alternatively, the steps, discussed above, for tracking andreporting on marketing personnel training and attendance may beautomatically performed by the CMO digital twin 8308, requiring noaction from the user. References to features and functions of the EMPand digital twins in this example of a marketing department and a CMOdigital twin 8308 should be understood to apply to other departments anddigital twins, and their respective projects and workflows, except wherecontext indicates otherwise.

In embodiments, a CMO digital twin 8308 may be configured to interfacewith the collaboration suite 8006 to specify and provide a set ofcollaboration tools that may be leveraged by the marketing departmentand associated parties. The collaboration tools may include videoconferencing tools, “in-twin” collaboration tools, whiteboard tools,presentation tools, word processing tools, spreadsheet tools, and thelike, as described herein. Collaboration and communication rules may beconfigured based at least in part on using the AI reporting tool, asdescribed herein.

In embodiments, a CMO digital twin 8308 may be configured to research,create, track and report on a marketing department budget including, butnot limited to, an overall department budget, a budget for a single orgroup of marketing or advertising campaigns, a budget for a third-partyvendor, or some other type of budget. The CMO digital twin 8308 mayinteract with and share such budget data and reporting with otherexecutive twins, as described herein, including, but not limited to, adigital twin related to the finance department, accounts payable,executive staff such as the CEO and CFO, or others. The CMO digital twin8308 may include intelligence, based at least in part on the dataanalytics, machine learning and A.I. processes, as described herein, toread marketing budgets and related summaries and data in order toidentify key departments, personnel, third-party or others that are, forexample, listed in, or subject to, the budget line item and whotherefore may have an interest in such material. Budget materialpertaining to a given party may be abstracted and summarized forpresentation independent from the entirety of the budget, and formattedand presented automatically, or at the direction of a user, to the partythat is the subject of the budget item. In a simplified example, a CMOmay create a new marketing campaign, “Airline—Airfare coupon textingcampaign—January,” which includes the following line items: Third-partyadvertising firm content creation $15,000; Social media platformplacement $50,000; analytics department $25,000, and so forth. Theentirety of the budget may be shared (at the election of the user orautomatically) with parties that must approve the full budget, such as aCFO. As described herein this sharing may be accomplished by the CMOdigital twin 8308 communicating directly with a CFO digital twin, sothat the information is presented to the CFO without requiring the CFOto have knowledge of the budget or requesting the budget. Subparts ofthe budget, for example, the analytics department line item, may beautomatically sent to the head of the analytics department by the CMOdigital twin 8308 to inform that department of the total amount ofauthorized spending that is approved for that department for thespecific marketing campaign.

In embodiments, a CMO digital twin 8308 may be configured to track andreport on inbound and outbound billing (i.e., accounts receivable andpayable) related to the marketing department. The billing department,personnel, processes and systems, including a Billing digital twin mayinteract with the CMO digital twin 8308 to present, store, analyze,reconcile and/or report on billing activities related to parties withwhom the marketing department is contracting, such as ad agencies, adnetworks, ad exchanges, content creators, advertisers, social mediaplatforms, television, radio, online entities, or others.

In embodiments, a CMO digital twin 8308 may be configured to depictmarketing campaign twins. In these embodiments, the CMO digital twin8308 may depict various states and/or items relating to a markingcampaign such as marketing content associated with a marketing campaign,market research performed with respect to a marketing campaign, trackingdata of marketing content associated with marketing campaigns (e.g.,geographic reach of marketing campaigns, demographic data associatedwith campaigns, etc.), analyses of marketing campaigns (e.g., outcomesrelated to marketing campaigns on various platforms), and the like. Insome embodiments, a CMO digital twin may be configured to automaticallyreport on marketing campaign-related activity via a user interfaceassociated with the CMO digital twin 8308. Such activities may bedetermined using marketing department metadata that indicates statechanges, such as an alteration to a website content, a change to aproduct photograph in an advertisement, a change in wording of amailing, and the like. The CMO digital twin 8308 may also depictactivity among a class of entities that are monitored or that arespecified for monitoring in the CMO digital twin 8308, such as a newpress release regarding a discounted advertising opportunity availablefrom an ad exchange. In embodiments, a CMO digital twin 8308 may beconfigured to provide research, tracking, monitoring, and analyses ofmedia content performance across various marketing related platforms,and automatically report on such activity to a user interface associatedwith the CMO digital twin 8308. Such platforms may include, but are notlimited to, customer relationship platforms (CRMs), organizationwebsite(s), social media, blogs, press releases, mailings, in-store orother promotions, or some other type of marketing platform-relatedmaterial or activity.

In some of these embodiments, the CMO digital twin 8308 may beconfigured to simulate marketing campaigns, such that the simulations ofthe marketing campaign may vary parameters such as vehicles (e.g.,social media, television, billboards, print, etc.), budget, targetingparameters (e.g., geographic, demographic, or the like), and/or othersuitable marketing campaign parameters. In these embodiments, thedigital twin simulation system 8116 may receive a request to perform thesimulation CMO digital twin, where the request indicates campaignfeatures and the parameters that are to be varied. In response, thedigital twin simulation 8116 may return the simulation results to theCMO digital twin 8308, which in turn outputs the results to the user viathe client device display. In this way, the user is provided withvarious outcomes corresponding to different parameter configurations. Insome embodiments, the user may select a parameter set based on thevarious outcomes. In some embodiments, an executive agent trained by theuser may select the parameter sets based on the various outcomes.

In embodiments, a CMO digital twin 8308 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to a marketing strategy, plan, campaign or initiative. Forexample, the CMO digital twin 8308 may be associated with a plurality ofdatabases or other repositories of marketing presentation materials,summaries and reports and analytics, including such presentationmaterials, summaries and reports and analytics related to priormarketing campaigns, each of which may be further associated withfinancial and performance metrics pertaining to the campaign and whichare also accessible to the CMO digital twin 8308. Such historicalmarketing campaign material may consist of advertising, marketing orother content that may be categorized based in part on the financial andperformance metrics with which it is associated. For example, there maybe a first category called “Market Tested Content,” which consists ofcontent that has been field deployed in a marketing campaign within acustomer population, the actual performance of which is therefore fullyknown based on actual market testing. Because the marketing content fromthis category has been field tested, the content may be scored based atleast in part on the financial, performance or other data with which itis associated. A second category may be “New Content—Simulation Tested,”which consists of content that has not been deployed in the field, butwhich has been subject to analytic testing such as simulated customersegmentation analysis, simulated A/B testing, simulated attributionmodeling, simulated market mix modeling, machine learning, A.I.techniques including, but not limited to, classification, probabilisticmodeling, learning techniques, and the like. Because the marketingcontent from this category has been simulation tested, the content maybe scored based at least in part on the simulated performance data orother data with which it is associated. Continuing the example, a thirdcategory of content may be “New Content—Panel Tested,” which consists ofcontent that has not been deployed in the field, nor simulation tested,but which has been subject to testing among a human panel for theirviews, opinions and impressions. Because the marketing content from thiscategory has been human panel tested, the content may be scored based atleast in part on the performance data, as reported by the human panel,or other data with which it is associated. A final, fourth category ofcontent may be “New—Untested,” which is newly developed or other contentthat has not been tested in the field, in simulation, or by a humanpanel. The CMO digital twin 8308 may utilize the machine learning, A.I.and other analytic capabilities, as described herein, to analyze thecontent of the four categories of content and classify and score thecontent characteristics that are probabilistically associated withimproved financial or other performance for stated types of marketingcampaigns or marketing subject matter. Statistical weights may beapplied to such characteristics, where the weight is indicative of agreater degree of financial or some performance metric of interest.Similarly, the characteristics of the market may be analyzed vis-a-visthe marketing content to determine the consumer characteristics that areprobabilistically associated with improved financial or otherperformance for given marketing content. The CMO digital twin 8308 mayprovide a user interface within which access to this repository ofstored data on content category, consumer and performance is available.When planning a marketing campaign, the CMO, or other marketingpersonnel, may use the CMO digital twin 8308 to select from thisrepository of content, that content which probabilistically will performbetter with the intended consumer targets of the new campaign. Forexample, from historical marketing field tests from actual priormarketing campaigns, the data may show that marketing content havingimages of large dogs outperformed (based on, for example, ad conversionrates) content picturing small dogs, and this effect was positivelycorrelated with age (i.e., older persons have an even greater preferencefor larger dogs). The performance data from the simulation-testedcontent may show a similar, but smaller effect based on the size of thedog images in the content, and the panel-tested data may show a similareffect for large dog imagery in content, but also have performance dataindicating that the effect appears, based on the panel data, to be mutedfor persons 15 years or younger (i.e., young persons are more attractedto smaller dog breeds than older persons). For the CMO using the CMOdigital twin 8308 this data, and the characteristics of the moresuccessful content, may be used to select from the fourth category ofcontent (“New—Untested”) that content that is most appropriate for a newmarketing campaign intended to sell a soft drink. In embodiments, theartificial intelligence services system 8010 of the EMP 8000 may selectthe content and segment its presentation based at least in part on theprior performance data, so that the ads that are presented on platformsthat tend to have persons over 15 will use content having a predominanceof large breed dogs, and those platforms with younger audiences willoffer a greater mix of dog breeds and possibly a preference for smallbreed dogs in marketing images. As the marketing campaign deployed tothe field, the CMO digital twin 8308 may monitor, track and report onthe marketing campaign's performance so that the CMO can review andintervene as necessary. Once the new content has been field tested itmay be stored and classified in the first category of content, “MarketTested Content,” along with the related financial and performancemetrics. In another example, similar stored content, content categories,characteristics and financial and performance metrics may be used by theCMO digital twin 8308 to recommend, for example, search engineoptimization (SEO), or other marketing strategies and techniques.

In embodiments, a CMO digital twin 8308 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to market surveys, online surveys, customer panels, ratings,rankings, marketing trend data or other data related to marketing. A CMOdigital twin 8308 may link to, interact with, and be associated withexternal data sources, and able to upload, download, aggregate externaldata sources, including with the EMP's internal data, and analyze suchdata, as described herein. Data analysis, machine learning, AIprocessing, and other analysis may be coordinated between the CMOdigital twin 8308 and an analytics team based at least in part on usingthe artificial intelligence services system 8010. This cooperation andinteraction may include assisting with seeding data elements and domainsin the enterprise data store 8012 for use in modeling, machine learning,and AI processing to identify the optimal marketing content, saleschannels, target consumers, price points, timing, or some othermarketing-relating metric or aspect, as well as identification of theoptimal data measurement parameters on which to base judgement of amarketing endeavor's success. Examples of data sources 8020 that may beconnected to, associated with, and/or accessed from the CMO digital twin8308 may include, but are not limited to, the sensor system 8022, thesales database 8024 that is updated with sales figures in real time, theCRM system 8026, the content marketing platform 8028, news websites, thefinancial database 8030 that tracks costs of the business, surveys 8032(e.g., customer satisfaction surveys), an org chart 8034, a workflowmanagement system 8036, customer databases 8040 that store customerdata, and/or third-party data sources 8038 that store third-party data.

In embodiments, a CMO digital twin 8308 may be configured to assist inthe development of a new marketing campaign. For example, the CMOdigital twin 8308 may identify an internal and external partner team fora marketing campaign. For example, individuals who are ideal candidatesto assist with a marketing campaign may be identified based at least inpart on experience and expertise data that is stored within or inassociation with the CMO digital twin 8308. In another example, the CMOdigital twin 8308 may identify marketing campaign goals and record,monitor and track the campaign's performance relative to those goals andpresent, in real-time, the tracking of the campaign to the CMO within auser interface that is associated with the CMO digital twin 8308.Examples of marketing targets include, but are not limited to, unitdistribution, customer acquisition customer retention, customer churn,customer loyalty (e.g., repeat purchases), customer acquisition costs,duration of average sales cycle, ad conversion rate, sales growth,geographic expansion of sales, demographic expansion of sales, marketpenetration, percentage of market control, marketing campaign ROI,regional comparison of performance, channel analysis, sales partneranalysis, marketing partner analysis, or some other marketing target.

In embodiments, a CMO digital twin 8308 may be configured to monitorcustomer feedback loops, customer opinions, customer satisfaction,complaints, product returns and the like based at least in part on useof the monitoring agent of the client application 8052, as describedherein, that is associated with the CMO digital twin 8308. Such feedbackdata may include, but is not limited to, data that derives from callcenter activity, chatbot activity, email (e.g., complaints), productreturns, Better Business Bureau submissions, or some other type ofcustomer feedback or manifestation of customer opinion. The clientapplication 8052 may include a monitoring agent that monitors the mannerby which customers or others respond to a marketing campaign. Themonitoring agent may report the customer's response to such campaigns tothe EMP 8000 for presentation in a user interface that is associatedwith the CMO digital twin 8308. In response, the EMP 8000 may train anexecutive agent (which may include one or more machine-learned models)to handle and process such notifications when they next arrive, andescalate and/or alert the CMO when such notifications are of an urgentnature, for example, an announcement of a class action lawsuit relatedto a product that is the subject of a marketing campaign. Inembodiments, the CMO digital twin 8308 may generate performance alertsbased on performance trends. This may allow a CMO to optimize marketingcampaigns in real-time without having to manually request such real-timeperformance data; the CMO digital twin 8308 may automatically presentsuch information and related/necessary alerts as configured by theorganization, CMO, or some other interested party.

In embodiments, a CMO digital twin 8308 may be configured to report onthe performance of the marketing department, personnel of the marketingdepartment, marketing campaigns, marketing content, marketing platforms,marketing partners, or some other aspect of management within a CMO'spurview. Reporting may be to the CMO, the marketing department, to otherexecutives of an organization (e.g., the CEO), or to outside thirdparties (e.g., marketing partners, press releases, and the like). Asdescribed herein, reporting may include sales summaries, customer data,marketing campaign performance metrics, cost-per-sale data,cost-per-conversion data, customer analysis, such as predicted customerlifetime value for newly acquired customers, or some other type ofreporting data. Reporting and the content of reporting may be shared bythe CMO digital twin 8308 with other executive digital twins, forexample, data related to new customers having a particularly highpredicted customer lifetime value may be shared with a sales staff forthe purpose of exploring cross-selling opportunities. The reportingfunctionality of the CMO digital twin 8308 may also be used forpopulating required data for formal reporting requirements such asshareholder statements, annual reports, SEC filings, and the like.Templets of common reporting formats may be stored and associated withthe CMO digital twin 8308 to automate the presentation of data andanalytics according to pre-defined formats, styles and systemrequirements

In embodiments, a CMO digital twin 8308 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to competitors of a CMO's organization, or namedentities of interest. In embodiments, such data may be collected by theEMP 8000 via data aggregation, spidering, web-scraping, or othertechniques to search and collect competitor information from sourcesincluding, but not limited to, press releases, SEC or other financialreports, mergers and acquisitions activity, or some other publiclyavailable data.

In embodiments, a CMO digital twin 8308 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to regulatory activity, such as governmentregulations, industry best practices or some other requirement orstandard. For example, the marketing industry is subject to data privacyand security laws in many jurisdictions, and it is an area of law andregulation that is experiencing rapid change. In embodiments, the CMOdigital twin 8308 may be in communication with another enterprisedigital twin, such as a General Counsel digital twin 8314, through whichthe legal team can keep the CMO apprised of new regulation or regulationchanges as they occur. Similarly, as a CMO develops new market campaignsand selects the jurisdictions (e.g., United States vs Europe) andpopulations that will be a part of the campaigns (e.g., minors vs.adults), the CMO digital twin 8308 may automatically send a synopsis ofthe aspects of the campaigns that are relevant for privacy law review sothat the campaign may be vetted for legal and regulatory complianceprior to launch. In an example, such a marketing campaign synopsis mightinclude a summary of the jurisdictions of the campaign, intendedaudience, means of obtaining consent, the type of consent to be obtained(e.g., opt-in, opt-out, passive), and so forth. Once approved andlaunched, as customer consents and other data privacy-relatedinformation is received by an organization, the CMO digital twin 8308may facilitate the CMO tracking metrics, for example the percentage ofcustomers choosing to opt-in to receive future marketing material (e.g.,email solicitations). As the organization receives privacy relatedmaterial it may store such information for future retrieval, summary,deletion or other activity, for example, in response to a data subjectrequest from an EU citizen who has requested their data be deleted(i.e., exercising their “right to be forgotten”). In embodiments, theCMO digital twin 8308 may monitor, store, aggregate, merge, analyze,prepare, report and distribute material relating to what customer datais collected, the party responsible for its collection and storage, thelocation and duration of storage, and so forth. This data may be calledforth by the CMO digital twin 8308, for example, in the event of a databreach. The CMO digital twin 8308 may be able to summarize, for example,a list of persons affected by the breach and the type of data that wasbreached and share this information with a Chief Privacy Officer (CPO),including sharing with the CPO digital twin.

In embodiments, the client application 8052 that executes the CMOdigital twin 8308 may be configured with an executive agent that reportsa CMO's behaviors and preferences (or other marketing personnel'sbehaviors and preferences) to the expert agent system 8008, as describedherein, and the expert agent system 8008 may train the executive agenton how the CMO or other marketing personnel respond to certainsituations and adjust its operation based at least in part on the datacollection, analysis, machine learning and A.I. techniques, as describedherein.

In embodiments, a Chief Technical officer (CTO) digital twin 8310 may bea digital twin configured for a CTO or other technology executive of anenterprise tasked with overseeing and managing the R&D, technologydevelopment, technical implementations of the enterprise, and/orengineering activities of the enterprise. In embodiments, the CTOdigital twin 8310 provides real-time views of enterprise technologyassets, including technology capabilities and versions. For example, ina manufacturing enterprise, a CTO digital twin 8310 may depict whereenvironment-compatible updates, upgrades, or substitutions may beavailable. A CTO digital twin 8310 may provide data, analytics, summary,and/or technical reporting including, but not limited to, real-time,historical, aggregated, comparison, and/or forecasted technicalinformation (e.g., real-time, historical, simulated, and/or forecastedtechnical performance data related to company products, benchmarkingresults, and the like). A CTO using by a CTO digital twin 8310 may bebetter able to stay abreast of technical developments and softwareengineering impacts by engaging in continuous virtualized learning usingthe CTO digital twin 8310. In embodiments, the CTO digital twin 8310 mayassist in virtual collaboration (a CTO-essential skill), as a CTO willneed to partner with in-house engineers and external vendors in avirtual environment to imagine and ideate to achieve something, oftensomething that hasn't been done before. In embodiments, the CTO digitaltwin may work in connection with the EMP 8000 to provide simulations,predictions, statistical summaries, decision support based on analytics,machine learning, and/or other AI and learning-type processing of inputs(e.g., technical performance data, sensor data and the like).

In embodiments, the CTO digital twin 8310 may provide features andfunctionality including, but not limited to, management of technicalpersonnel, partners and outside consultants and contractors (e.g.,developers, beta testers, and the like), oversight of budgets,procurement, expenditures, policy compliance (e.g., policies related tocode usage, storage, documentation, and the like), and other technology,development, and/or engineering-related resources, and/or reporting.

In embodiments, the types of data that may populate a CTO digital twinmay include, but are not limited to, technology performance andspecification data, interoperability and compatibility data,cybersecurity data, competitor data, failure mode effects analysis(FMEA) data, technology/engineering roadmap data, information technologysystems data (including with respect to any of the hardware, software,networking, and other types mentioned or described herein), operationstechnology and systems data, uptime/downtime/operational performancedata, asset aging/vintage/timing data, technical performance metrics bybusiness unit, by product, by geography, by factory, by storelocation(s), resource utilization, competitive product and pricing data,forecast data, demand planning data, analytic results of AI and/ormachine learning modeling (e.g., technical forecasting), predictiondata, metrics relating to patent disclosures, patent filings, and/orpatent grants, recommendation data, and/or other types of data relevantto the operations of the CTO and/or technology, development, and/orengineering department.

In embodiments, the CTO digital twin 8310 may depict a twin of a set oftechnology, development, and/or engineering departments, which the usermay use to identify, assign, instruct, oversee and review technology,development, and/or engineering department personnel and third-partypersonnel that are associated with the technology, development, and/orengineering activities of an organization, including third-partypartners and other outside contractors, such as third-party developersand/or testers that are involved in the organization's technology,development, and/or engineering activities. Examples of suchorganization personnel include, but are not limited to, technology,development, and/or engineering department staff, sales staff andanalysts, statisticians, data scientists, or some other type oforganization personnel relevant to the functioning of a technology,development, and/or engineering department. Examples of a technology,development, and/or engineering department's third-party personnelinclude, but are not limited to, management consultants, developers,software engineers, testers, and/or engineering partners, consultants,contractors, technical firm staff, auditors, or some other type ofthird-party personnel.

In embodiments, the CTO digital twin 8310 may include a definition ofthe various roles/employees working under the CTO, the reportingstructure, and associated permissions, for each individual in thebusiness unit, and may be populated with the various names and/or otheridentifiers of the individuals filling the respective roles.

In embodiments, the client application 8052 executing a CTO digital twin8310 may interface with the collaboration suite 8006 to specify andprovide a set of collaboration tools that may be leveraged by thetechnology, development, and/or engineering department and associatedparties. The collaboration tools may include video conferencing tools,“in-twin” collaboration tools, whiteboard tools, presentation tools,word processing tools, spreadsheet tools, and the like, as describedherein. Collaboration and communication rules may be configured based atleast in part on using the AI reporting tool, as described herein.Collaboration and communication tools and associated rules may beconfigured to use company-, industry- and domain-specific taxonomies andlexicons when representing entities, states and flows within the CTOdigital twin 8310.

In embodiments, the CTO digital twin 8310 may be configured to allow auser to research, create, track and report on a technology, development,and/or technology or engineering department initiative including, butnot limited to, a new product development, update, enhancement,replacement, upgrade, or the like. In embodiments, the CTO digital twin8310 may be associated and/or in communication with databases, includingdatabases storing analytic and/or product data and product performancedata, and present information to an interface associated with the CTOdigital twin 8310, as described herein. As product development advances,real time operations and other technical information may be used tocontinuously update the product development summary that is availablefor the CTO or other technical personnel to review. The CTO digital twin8310 may also be associated and/or in communication with databases,including databases storing analytic and/or competitive product data andproduct performance data, and present this information to an interfaceassociated with the CTO digital twin 8310, as described herein. As theCTO's company's products change, and competitor products change, theircurrent state and specifications may be presented by the CTO digitaltwin 8310 for the CTO or other technical personnel to review directproduct comparisons. Such comparisons may be used, in part, to produceanalytics, scores, reports and the like indicating the relativeadvantages and/or disadvantages that a company's product(s) has relativeto competitor product(s). In an example, a report may be automaticallyprovided to the marketing department to emphasize the relativeadvantages that a company product has over a competitor product (e.g.,speed of processing) that should be used in a new marketing campaign.Sharing with the marketing department may be accomplished, in part, bythe CTO digital twin 8310 communicating with the CMO digital twin 8308to present reports or other information to the CMO or marketing staff.

In embodiments, the CTO digital twin 8310 may be configured to presentsimulations of technology development and/or engineering activities. Forexample, in some embodiments, the digital twin system 8004 may simulateproduct usage under a plurality of constraints that might impact productperformance, such as an operating environment, processing speed, storageor other platform characteristics. In embodiments, real time operationsdata, such as operations data available through the EMP 100, may beincorporated into simulated data for the purposes of running operationalsimulations. This may allow a CTO to a gain a deeper understanding ofthe operation of the company's products in the real world and within analtered, simulated real world environment. It may also allow operationaldigital twin-based product architectures to be built that link actualproduct production with business priorities to enable simulated decisionmaking in a virtual environment and assist in the evaluation of vendorsupplied solutions by enabling the review of such digital twins in thecontext of their supplied solutions and the relationship to thebusiness. In embodiments, simulations may also include simulationsrelated to varying technical and/or product specification parameters,product design and monitoring, internal controls design, testing,certification, and deliver technical and non-technical data in reports,presentations, and dashboards for technical decision making. In theseembodiments, the digital twin simulation system 8116 may receive arequest to perform the simulation requested by the CTO digital twin8310, where the request indicates features and the parameters, includingtechnical parameters, that are to be varied. In response, the digitaltwin simulation system 81D16 may return the simulation results to theCTO digital twin 8310, which in turn outputs the results to the user viathe client device display. In this way, the user is provided withvarious outcomes corresponding to different technical and/or productparameter configurations. In some embodiments, the user may select aparameter set based on the various outcomes. In some embodiments, anexecutive agent trained by the user may select a technical parameter setbased on the various outcomes. The simulations, analytics and/ormodeling performed by the CTO digital twin 8310 may be used to reducetesting time, design time, or some other type of technical cost. Thesimulations, analytics and/or modeling performed by the CTO digital twin8310 may be used to create and structure product development and testingplans. The simulations, analytics and/or modeling performed by the CTOdigital twin 8310 may be used to evaluate product go-to-market timingand preparedness. The CTO equipped with a CTO digital twin 8310 will bebetter able to adapt quickly to identify product and/or technicalparameters in need of further development and predict products'operational performance. This may reduce errors, speed testing andreduce the need for patches, bug fixes, updates and the like and flattenagile process management.

In embodiments, the CTO digital twin 8310 may provide an interface thatallows a user to research, create, track and report on a technology,development, and/or engineering department initiative including, but notlimited to, an overall department budget, a budget for a single or groupof technology, development, and/or engineering initiatives, athird-party vendor activity, or some other type of expense or budget.The CTO digital twin 8310 may interact with and share such expense orbudget data and reporting with other executive twins, including, but notlimited to, a digital twin related to accounts payable, executive staffsuch as the CEO, and/or others.

In embodiments, the CTO digital twin 8310 may leverage the artificialintelligence services system 8010 (e.g., data analytics, machinelearning and A.I. processes) to read technical reports, projections,simulations, and related summaries and data in order to identify keydepartments, personnel, third-party or others that are, for example,listed in, or subject to, a technical item or detail provided.

In embodiments, the CTO digital twin 8310 may be configured to provide aCTO, or other technology, development, and/or engineering departmentpersonnel, with information that is unique to the CTO digital twin 8310and thus can provide insights and perspectives on technical performancethat are unique to the CTO digital twin 8310, based at least in part onthe CTO digital twin 8310 make making use of real time production,development and operational data based on both real world and simulatedactivity.

In embodiments, the CTO digital twin 8310 may be configured to manageoperational planning, based at least in part by leveraging predictiveanalytics for development planning, and supply chain management in orderto increase company efficacy while optimizing operating expenses. Inembodiments, the CTO digital twin 8310 may be configured to obtain anddepict oversight activity that includes, but is not limited to, internalcontrols design, testing, and reporting while directing listed actionsthe appropriate personnel.

In embodiments, the CTO digital twin 8310 may be configured to depict,aggregate, merge, analyze, prepare, report and distribute materialrelating to a technical strategy, plan, activity or initiative. Forexample, the CTO digital twin 8310 may be associated with a plurality ofdatabases or other repositories of technical materials, summaries andreports and analytics, including such materials, summaries and reportsand analytics related to prior technical activity and results (e.g., bugtesting), each of which may be further associated with third-partytechnical or economic data, including competitor product data and/ortechnical benchmarks.

In embodiments, the CTO digital twin 8310 may be configured to depict,aggregate, merge, analyze, prepare, report and distribute materialrelating to technical reporting, ratings, rankings, technical trenddata, or other data related to company technology, development, and/orengineering. A CTO digital twin 8310 may link to, interact with, and beassociated with external data sources, and able to upload, download,aggregate external data sources, including with the EMP's internal data,and analyze such data, as described herein. Data analysis, machinelearning, AI processing, and other analysis may be coordinated betweenthe CTO digital twin 8310 and an analytics team based at least in parton using the intelligence services system 8010. This cooperation andinteraction may include assisting with seeding technology, development,and/or engineering-related data elements and domains in the enterprisedata store 8012 for use in modeling, machine learning, and AI processingto identify the optimal technical strategy, or some other technology,development, and/or engineering-relating metric or aspect, as well asidentification of the optimal data measurement parameters on which tobase judgement of a technology initiative, development initiative,and/or engineering endeavor's success. Examples of data sources 8020that may be connected to, associated with, and/or accessed from the CTOdigital twin 8310 may include, but are not limited to, the sensor system8022, the sales database 8024 that is updated with sales figures in realtime, a technology, development, and/or engineering platform, newswebsites 8048, a technical database that tracks costs of the business,an org chart 8034, a workflow management system 8036, customer databases8040 that store customer data, and/or third-party data sources 8038 thatstore third-party data.

In embodiments, the CTO digital twin 8310 may aggregate data sources andtypes, creating new data types, summaries and reports that are notavailable elsewhere. This may reduce reliance upon the need of multiplethird-party providers and current solutions. This may, among otherbenefits and improvements, reduce expenses associated with acquiringdata needed for sound technical decision making.

In embodiments, the CTO digital twin 8310 may be configured to monitortechnical performance, including real time monitoring, based at least inpart on use of the monitoring agent of the client application 8052, asdescribed herein, that is associated with the CTO digital twin 8310. Themonitoring agent may report on such activities to the EMP 8000 forpresentation in a user interface that is associated with the CTO digitaltwin 8310. In response, the EMP 8000 may train an executive agent (whichmay include one or more machine-learned models) to handle and processsuch notifications when they next arrive, and escalate and/or alert theCTO when such notifications are of an urgent nature, for example, anidentification of a new technical bug or a security patch that isurgently needed. In embodiments, the CTO digital twin 8310 may generatetechnical performance alerts based on performance trends. This may allowa CTO to optimize initiatives in real-time without having to manuallyrequest such real-time technical performance data; the CTO digital twin8310 may automatically present such information and related/necessaryalerts as configured by the organization, CTO, or some other interestedparty.

In embodiments, the CTO digital twin 8310 may be configured to report onthe performance of the technology, development, and/or engineeringdepartment, personnel of the technology, development, and/or engineeringdepartment, technology, development, and/or engineering activities,technology, development, and/or engineering content, technology,development, and/or engineering platforms, technology, development,and/or engineering partners, or some other aspect of management within aCTO's responsibilities. Reporting may be to the CEO, the technology,development, and/or engineering department, to other executives of anorganization (e.g., the CIO), or to outside third parties.

In embodiments, the CTO digital twin 8310 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to industry best practices, benchmarks, or some otherrequirement or standard. For example, the CTO digital twin 8310 may bein communication with another enterprise digital twin, such as a CIOdigital twin 8312, through which the technical team can keep the CIOapprised of changes as they occur.

In embodiments, the client application 8052 that executes the CTOdigital twin 8310 may be configured with an executive agent that reportsa CTO's behaviors and preferences (or other technology, development,and/or engineering personnel's behaviors and preferences) to theexecutive agent system 8008, as described herein, and the executiveagent system 8008 may train the executive agent on how the CTO or othertechnology, development, and/or engineering personnel respond to certainsituations and adjust its operation based at least in part on the datacollection, analysis, machine learning and A.I. techniques, as describedherein.

References to features and functions of the EMP and digital twins inthis example of the CTO digital twin 8310 should be understood to applyto other departments and digital twins, and their respective projectsand workflows, except where context indicates otherwise.

In embodiments, a Chief Information Officer (CIO) digital twin 8312 maybe a digital twin configured for the CIO of an enterprise, or analogousexecutive tasked with overseeing the intelligence, information, data,knowledge, and/or IT operations of the enterprise. In embodiments, a CIOdigital twin 8312 depicts a real time representation of anorganization's information assets and workflows including data relatingto data security, network security and enterprise knowledge. The realtime representation may be based at least in part on real-timeoperations data that tracks the performance of an organization'sinformation infrastructure, including internal information assets,customer-facing technologies, and information assets provided and/orserviced by third parties, such as cloud computing service providers.For example, a CIO digital twin 8312 may receive real time informationregarding the performance of a network, such as an intranet used by anorganization, APIs that are accessed by the enterprise, APIs that areexposed by the enterprise, software that is running on the enterprisessoftware, or the like. The information may be aggregated and presentedto a CIO in order to provide him an overview of the general performanceof the computing infrastructure of the enterprise. For example, the CIOdigital twin may indicate whether there are any network outagesoccurring, whether there are any security risks detected in theenterprises network, whether any software systems are operatingimproperly, and may other scenarios. In embodiments, the CIO digitaltwin 8312 may present a user interface that allows a user (e.g., theCIO) to select particular network assets to review in greater detail,such as an asset the real time operations data indicates is experiencingan operational failure or other issue. Such real time operations datarelated to IT and other information asset performance may allow the CIOto better track the performance and needs of an organization'sinformation and IT infrastructure and better enable him to troubleshootissues, simulate solutions, select appropriate information and ITmanagement actions, and maintain the organization's information and ITinfrastructure.

In embodiments, a CIO digital twin 8312 may provide data, analytics,summary, and/or information and IT reporting including, but not limitedto, real-time, historical, aggregated, comparison, and/or forecastedinformation (e.g., real-time, historical, simulated, and/or forecastedperformance data related to company information and IT assets,third-party assets, and the like). A CIO empowered by a CIO digital twin8312 may be better able to maintain and evolve information and IT assetsthrough continuous monitoring using the CIO digital twin 8312. A CIOdigital twin 8312 may assist in virtual monitoring and testing in avirtual environment to test implementations, changes, reconfigurations,the introduction and/or removal of components and other assets, and thelike. In embodiments, the CIO digital twin may work in connection withthe EMP 8000 to provide simulations, predictions, statistical summaries,decision support based on analytics, machine learning, and/or other AIand learning-type processing of inputs (e.g., performance data, sensordata, and the like).

In embodiments, the types of data that may populate a CIO digital twin8312 may include, but are not limited to, information and IT assetperformance and specification data, interoperability and compatibilitydata, cybersecurity data, uptime/downtime/operational performance data,asset aging/vintage/timing data, resource utilization, results of AIand/or machine learning modeling (e.g., IT performance simulations), orsome other type of data relevant to the operations of the CIO.

In embodiments, a CIO digital twin 8312 may be configured to interfacewith the collaboration suite 8006 to specify and provide a set ofcollaboration tools that may be leveraged by the technology,development, and/or engineering department and associated parties. Thecollaboration tools may include video conferencing tools, “in-twin”collaboration tools, whiteboard tools, presentation tools, wordprocessing tools, spreadsheet tools, and the like, as described herein.Collaboration and communication rules may be configured based at leastin part on using the AI reporting tool, as described herein.Collaboration and communication tools and associated rules may beconfigured to use company-, industry- and domain-specific taxonomies andlexicons when representing entities, states and flows within the CIOdigital twin 8312.

In embodiments, the CIO digital twin 8312 may be configured to providesimulations of an organization's information and IT activitiesincluding, but not limited to network utilization, disaster planning, ITasset selection, maintenance protocols, downtime planning, and the likethat is simulated under a plurality of hypothetical IT environments andscenarios that might impact performance, such as a security breach, ITasset failure, information failure, network congestion, or otheractivity or event. Real time operations data, such as that availablethrough the EMP, as described herein, may be incorporated into simulatedinformation or IT Infrastructure scenarios for the purposes of runningoperational simulations. The simulations, analytics and/or modelingperformed by the EMP 100 with respect to a CIO digital twin 8312 may beused to reduce testing time, design time, or some other type of IT cost.The simulations, analytics and/or modeling performed by the CIO digitaltwin 8312 may be used to create and structure IT assets, networks, andguide development and testing plans. The simulations, analytics and/ormodeling performed by the CIO digital twin 8312 may be used to evaluatenetwork security, performance, and other features. The CIO equipped withdigital twin 8312 may quickly identify optimal asset configurations tomaximize operational performance.

In embodiments, a CIO digital twin 8312 may be configured to provide auser (e.g., the CIO) with information that is unique to the CIO digitaltwin 8312 and thus can provide insights and perspectives on informationand IT asset performance that are unique to the CIO digital twin 8312,based at least in part on the CIO digital twin 8312 make making use ofreal time production, development and operational data based on bothreal world and simulated activity. In embodiments, the CIO digital twin8312 may be configured to manage operational planning, based at least inpart by leveraging predictive analytics for development planning. Inembodiments, a CIO digital twin 8312 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to an information and/or IT strategy, scenario, event, plan,activity or initiative. For example, the CIO digital twin 8312 may beassociated with a plurality of databases or other repositories ofinformation, materials, summaries and reports and analytics, includingsuch materials, summaries and reports and analytics related to priorevents, activity and results (e.g., a system outage).

In embodiments, a CIO digital twin 8312 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to information and/or IT reporting, ratings, rankings,information, knowledge and IT trend data, or other data related tocompany information and/or IT assets and infrastructure. A CIO digitaltwin 8312 may link to, interact with, and be associated with externaldata sources, such that the CIO digital twin 8312 may upload, download,aggregate external data sources, and/or analyze such enterprise data.

In embodiments, a CIO digital twin 8312 may be configured to monitor ITperformance, including in real time, based at least in part on use ofthe monitoring agent of the client application 8052, as describedherein, that is associated with the CIO digital twin 8312. Themonitoring agent may report on such activities to the EMP 8000 forpresentation in a user interface that is associated with the CIO digitaltwin 8312. In response, the EMP 8000 may train an executive agent (whichmay include one or more machine-learned models) to handle and processsuch notifications when they next arrive and escalate and/or alert theCIO when such notifications are urgent.

In embodiments, a CIO digital twin 8312 may be configured to report onthe performance of an organization's IT assets, network, or some otheraspect of management within a CIO's responsibilities. In embodiments,the client application 8052 that executes the CIO digital twin 8312 maybe configured with an executive agent that reports a CIO's behaviors andpreferences to the executive agent system 8008, and the executive agentsystem 8008 may train the executive agent on how the CIO or otherpersonnel respond to certain IT situations and adjust its operationbased at least in part on the data collection, analysis, machinelearning and A.I. techniques described throughout the disclosure.

References to features and functions of the EMP and digital twins inthis example of a marketing department and a CIO digital twin 8312should be understood to apply to other departments and digital twins,and their respective projects and workflows, except where contextindicates otherwise.

In embodiments, a general counsel (GC) digital twin 8314 may be anexecutive digital twin configured for the general counsel (GC) of anenterprise, or an analogous executive tasked with overseeing the legaldepartment and/or outside counsel of the enterprise. A GC digital twin8314 may provide functionality including, but not limited to, managementof legal personnel, partners and outside counsel, oversight of legalbudgets and resources, compliance, management of contracting andlitigation, management of internal policies, intellectual property,employment law, tax law, privacy law, reporting, and regulatoryanalysis.

In embodiments, the types of data that may populate and/or be utilizedby a GC digital twin 8314 may include, but are not limited to, budgetarydata (e.g., external legal spend, internal legal spend, ancillary legalcosts, and the like), regulatory data (e.g., regulatory requirements,regulatory actions taken, and the like); contract and licensing data(e.g., in progress negotiations, current contract obligations, pastcontract obligations, and the like); compliance data (e.g., compliancerequirements, compliance actions taken, and the like, litigation data(e.g., potential litigations sources, pending litigations, pastlitigations, settlement agreements, and the like), employment data(e.g., employment contracts, employee complaints, employee stockoptions, and the like), intellectual property data (e.g., filed patentapplications, patent dockets, issued patents, trademark applications,trademark docket data, registered trademarks, and the like), tax data,privacy data, regulatory data, analytic results of AI and/or machinelearning modeling; prediction data; recommendation data, or some othertype of data relevant to the operations of the GC and/or legaldepartment.

In embodiments, a GC digital twin 8314 may be configured based at leastin part on using the collaboration suite 8006 to specify and provide aset of collaboration tools that may be leveraged by the legal departmentand associated parties. The collaboration tools may include videoconferencing tools, “in-twin” collaboration tools, whiteboard tools,presentation tools, word processing tools, spreadsheet tools, and thelike, as described herein. Collaboration and communication rules may beconfigured based at least in part on using the AI reporting tool, asdescribed herein. Collaboration and communication tools and associatedrules may be configured to use company-, industry- and domain-specifictaxonomies and lexicons when representing entities, states and flowswithin the GC digital twin 8314, such as ones related to particularbodies of law, regulation, jurisdiction, or practice area, such as onesrelated to corporate law, commercial law, bankruptcy law, the law ofsecured transactions, banking law, customs law, export controlregulations, maritime law, trade law, international treaties, securitieslaw, contracts law, environmental law, international law, privacy law,data privacy law, patent law, civil and criminal procedure, trademarklaw, copyright law, trade secret law, unfair competition law, law oftorts, property law, advertising law, and many others.

In embodiments, a GC digital twin 8314 may be configured to research,create, track and issue reports on a legal department budget including,but not limited to, an overall department budget, a budget for aspecific project, such as “U.S. patent filings,” or group of projects, abudget for a specific litigation, a budget for a third-party vendor,such as outside counsel, or some other type of legal budget. A GCdigital twin 8314 may be configured to create, track, provide research,and report on financial data related to material under review orsupervisions of the legal department including, but not limited to,licensing revenues, licensing expenditures, or some other type offinancial data related to legal department review and responsibilities.In embodiments, the GC digital twin 8314 may interact with and sharesuch licensing revenue and/or budget data and reporting with otherexecutive twins, as described herein, including, but not limited to, aCFO digital twin 8304, CEO digital twin, COO digital twin, CTO digitaltwin, and the like. In embodiments, the GC digital twin 8314 may includeintelligence, based at least in part on the data analytics, machinelearning and A.I. processes, as described herein, to read legalcontracts, licenses, budgets and related summaries and data in order toidentify key departments, personnel, third-party or others that are, forexample, listed in, or subject to, or impacted by a license and/orbudget line item and who therefore may have an interest in suchmaterial. License and/or budget material pertaining to a given party maybe abstracted and summarized for presentation independent from theentirety of the budget, and formatted and presented automatically, or atthe direction of a user, to the party that is the subject of the budgetitem. In a simplified example, a GC may have license(s) under herdepartment's review which have line items, schedules, appendices and thelike detailing licensing revenues that will be owed to the organizationover a prescribed timeframe. The GC may use the GC digital twin 8314 toconsolidate, summarize and/or share such financial data derived, or tobe derived, from licensing revenues with another executive in anorganization, such as the CFO (e.g., via a CFO digital twin) and/or CEO(e.g., via a CEO digital twin). The data shared may indicate thelicensing revenues to be obtained in a given financial quarter to assistthe CFO and others in maintaining an accurate and current summary ofprojected quarterly revenues.

In embodiments, a GC digital twin 8314 may be configured to track andreport on inbound (e.g., settlement or litigation revenue) and outboundbilling (e.g., outside counsel costs) related to the legal department.The billing department, personnel, processes and systems may interactwith the GC digital twin 8314 to present, store, analyze, reconcileand/or report on billing activities related to parties with whom thelegal department is contracting, such as outside counsel, consultants,research services, online entities, or others. In embodiments, a GCdigital twin 8314 may be configured to research, track, monitor, store,analyze, create and distribute legal content, and automatically reporton such activity to a user interface associated with the GC digital twin8314. Such activities might include storing data so that the GC digitaltwin 8314 may detect a state change, for example, a new court filing ina litigation, a communication received from outside counsel, a newlicense draft from opposing counsel, a draft patent application, anotice from the United States Patent and Trademark Office, or some othertype of new or updated material. The GC digital twin 8314 may alsodetect activity among a class of entities that are monitored or that arespecified for monitoring in the GC digital twin 8314, such as particularcourts, regulatory or legislative bodies or some other type of entity.In embodiments, a GC digital twin 8314 may be configured to research,track, monitor, store, and analyze content of various legal relatedplatforms, and automatically report on such activity to a user interfaceassociated with the GC digital twin 8314. Such platforms may include,but are not limited to, bar or other legal associations, courts, legalsearch platforms, social media, legal blogs, press releases, or someother type of legal platform-related material or activity.

In embodiments, a GC digital twin 8314 may be configured to store,aggregate, merge, analyze, prepare, report and distribute materialrelating to a legal strategy, legal documents, litigation, legalrecommendations or some other legal activity. For example, the GCdigital twin 8314 may be associated with a plurality of databases orother repositories of legal materials, contracts, licenses, intellectualproperty (e.g., patent filings), summaries and reports and analytics. AGC digital twin 8314 may link to, interact with, and be associated withexternal data sources, and able to upload, download, aggregate externaldata sources, including with the EMP's internal data, and analyze suchdata, as described herein. Data analysis, machine learning, AIprocessing, and other analysis may be coordinated between the GC digitaltwin 8314 and an analytics team based at least in part on using theintelligence services system 8010. This cooperation and interaction mayinclude assisting with seeding data elements and domains in theenterprise data store 8012 for use in modeling, machine learning, and AIprocessing to identify the optimal and/or relevant legal content, legaldocuments, parties associated with a legal activity (e.g., alitigation), as well as identification of the optimal data measurementparameters on which to base judgement of a legal endeavor's success(e.g., licensing revenue, staying within a stated budget for the use ofoutside counsel, and the like). Examples of data sources 8020 that maybe connected to, associated with, and/or accessed from the GC digitaltwin 8314 may include, but are not limited to, a legal researchplatform, legal websites, news websites 8048, the financial database8030, contracts database, an HR database 8046, a workflow managementsystem 8036, and/or third-party data sources 8038 that store third-partydata.

In embodiments, a GC digital twin 8314 may be configured to assist inthe development of a new legal endeavor, such as pursuit of a newcontract, review of a new law or regulation impacting a business,litigation or arbitration, or some other legal activity. For example,the GC digital twin 8314 may identify an internal and external partner(e.g., outside counsel) team for a legal action. For example,individuals who are ideal candidates to assist with a legal action maybe identified based at least in part on experience and expertise datathat is stored within or in association with the GC digital twin 8314.For example, the GC may be initiating negotiations of a jointdevelopment agreement between entities that are located in the UnitedStates and Taiwan and may need to obtain outside Taiwanese counsel.Using the GC digital twin 8314, the GC may be presented with details ofprior outside counsel used in Taiwan for similar projects. In anotherexample, if the GC digital twin 8314 does not locate details of prioroutside counsel used in Taiwan for similar projects, the GC digital twin8314 may scan, research, collect and summarize information from publicor other sources on highly rated, recommended or other Taiwanese outsidecounsel that may be appropriate, based on skills, experience and thelike, to work on the joint development agreement project.

In embodiments, the GC digital twin 8314 may identify legal projectgoals and record, monitor and track the project's performance relativeto those goals and present, in real-time, the tracking of the project tothe GC within a user interface that is associated with the GC digitaltwin 8314. For example, the GC digital twin 8314 may include a clickabledashboard that, when clicked, illustrates the status of a set of legalprojects. In some embodiments, the dashboard may include timelines foreach project and a relative status of each project with respect to itstimeline.

In embodiments, a GC digital twin 8314 may be configured to report onthe performance of the legal department, personnel of the legaldepartment, legal actions, legal content, legal platforms, legalpartners, or some other aspect of a GC's management. Reporting may be tothe GC, the legal department, to other executives of an organization(e.g., the CEO), or to outside third parties (e.g., outside counsel,legal notices, press releases, and the like). Reporting and the contentof reporting may be shared by the GC digital twin 8314 with otherexecutive digital twins, for example, data related to regulationcompliance, ongoing litigation, or some other legal activity. Thereporting functionality of the GC digital twin 8314 may also be used forpopulating required data for formal reporting requirements such asshareholder statements, annual reports, SEC filings, and the like.Templates of common reporting formats may be stored and associated withthe GC digital twin 8314 to automate the presentation of data andanalytics according to pre-defined formats, styles and systemrequirements. In some embodiments, the GC digital twin may be configuredto leverage an executive agent 8364 trained on behalf of the GC tocreate and disseminate the reports.

In embodiments, a GC digital twin 8314 may be configured to monitor,store, aggregate, merge, analyze, prepare, report and distributematerial relating to regulatory activity, such as governmentregulations, regulatory compliance, legislation, court opinions,industry best practices or some other requirement or standard. Forexample, the GC digital twin 8314 may keep the GC apprised of newregulation or regulation changes as they occur. The GC may setparameters of the GC digital twin 8314 regarding the legal domains,subject matter areas, jurisdictions, or some other parameter, that areof interest to the GC that the GC digital twin 8314 should monitor.

In embodiments, a GC digital twin 8314 may leverage an executive agent8364 that is trained on user's (e.g., GC) behaviors and preferences (orother legal personnel's behaviors and preferences). In embodiments, theclient application 8052 hosting the GC digital twin 8314 may track theuser's actions relating to various events, notifications, alerts, or thelike and may report the tracked events using the expert agent system8008, as described herein. In response, the expert agent system 8008 maylearn how the GC or other legal personnel respond to certain situationsand may train an execute agent 8364 on behalf of the user (e.g., GC),such that the executive agent 8364 may respond to similar situationsonce deployed.

References to features and functions of the EMP and digital twins inthis example of a legal department and a GC digital twin 8314 should beunderstood to apply to other departments and digital twins, and theirrespective projects and workflows, except where context indicatesotherwise.

In embodiments, a Chief Human Resources Officer (CHRO) digital twin 8316(or HR digital twin 8316) is an executive digital twin configured for ahuman resources executive (e.g., a CHRO) of an enterprise or analogousexecutive tasked with overseeing the human resources HR aspects of theenterprise, such as a Chief People Officer (CPO), a chief talentofficer, a head of human resources, a director of human resources, orthe like. In embodiments, the CHRO digital twin 8316 may depictdifferent HR-related states of the enterprise, such as states relatingto human capital management, workforce management, risk management, andthe management of payroll, recruitment, regulatory compliance, employeeperformance, benefits, employee relations, time and attendance, trainingand development, compensation, onboarding, offboarding, successionplanning, and the like. In embodiments, the CHRO digital twin 8316 mayinitially depict the various states at a lower granularity level. A userthat is viewing the CHRO digital twin 8316 may select a state to drilldown into the selected state and view the selected state at a higherlevel of granularity.

In embodiments, the types of data that may be depicted in CHRO digitaltwin 8316 may include, but are not limited to: individual employee data,key performance indicators by business unit, key performance indicatorsby individual employee, risk management data, regulatory compliance data(e.g., OSHA and EPA compliance data), safety data, diversity data,benefits data (e.g., medical, dental, vision, and health savingsaccounts (HSA)) compensation data, compensation comparison data,compensation trend data, payroll data, overtime data, recruitment data,employee referrals data, applicant data, applicant screening data,applicant reference data, applicant background check data, offer data,time and attendance data, employee relations data, employee complaintsdata, onboarding data, offboarding data, employee training anddevelopment data, employee turnover rate data, voluntary employeeturnover rate data, new hire turnover rate data, high performer turnoverrate data, turnover rate by performance rating data, headcount and/orheadcount planning data (e.g., headcount to plan percentage), promotionrate data, succession plan data, organizational levels data, span ofcontrol data, employee survey data, cost to move employees belowmidpoint data, comparative ratio data, simulation data, decision supportdata from AI and/or machine learning systems, prediction data from AIand/or machine learning systems, classification data from AI and/ormachine learning systems, detection and/or identification data from AIand/or machine learning systems, and the like.

In embodiments, a CHRO digital twin 8316 may depict a data item with anicon indicating whether the data item is at a normal state, a suboptimalstate, a critical state, or an alarm state. In embodiments, the iconsmay be different colors, fonts, symbols, codes or the like. For example,a CHRO digital twin 8316 may depict high performer turnover rate datawith an orange icon indicating that the high performer turnover rate isat a critical level. Continuing the example, an HR executive may beenabled to escalate the high performer turnover rate data to anotherexecutive, such as the CEO, via the CHRO digital twin 8316. Inembodiments, a CHRO digital twin 8316 may automatically highlight dataitems that are at suboptimal, critical, or alarm state.

In embodiments, a CHRO digital twin 8316 may be configured to provide an“in-twin” collaboration suite having tools that may facilitatecommunication and collaboration between enterprise stakeholders. Inembodiments, the “in-twin” collaboration tools may include an interfaceenabling a user to escalate and/or deescalate data sets to another userassociated with the enterprise. In embodiments, the interface may beconfigured to enable a user to send a message with the data set,generate a request or assign a task related to the data set, and/orschedule an event associated with the data set. In embodiments, AIand/or machine learning could be leveraged to suggest message content,suggest event scheduling, suggest a request or task, and/or suggest arequest or task assignee. For example, an HR executive could escalate adata set related to employee training to the GC with a predictive textmessage about employee training and a calendar request at a timedetermined by AI and/or machine learning to attend a meeting related toemployee training. In embodiments, the “in twin” collaboration toolsinclude digital twin conferences. In embodiments, the “in twin”collaboration tools may include an “in-twin” messaging system and/or an“in-twin” video conferencing system for enabling enterprise stakeholdersto communicate. In embodiments, a machine learning and/or AI system maybe leveraged for automatically generating and/or assigning tasks fromthese communications. In embodiments, the “in-twin” videoconferencingsystem supports subchats. In embodiments, the subchats may be createdvia a “drag-and-drop” action in the user interface. In embodiments, the“in-twin” videoconferencing system may leverage machine learning and/orAI to make suggestions to optimize a user's lighting, audio, cameraplacement, and the like. In embodiments, the “in twin” videoconferencingsystem leverages machine learning and/or AI to automatically disable thevideo feed upon the detection of an inappropriate activity in the videofeed. In embodiments, the “in twin” collaboration suite includes an“in-twin” stakeholder approval system for collecting approval on actionsfrom other enterprise stakeholders. In embodiments, “in-twin”collaboration tools may include an AI-driven translation systemconfigured to intelligently translate communications amongst enterprisestakeholders to achieve maximum understanding by the user of the digitaltwin, wherein the AI driven translation system is configured totranslate from a first language to a second language (e.g., translateEnglish into a foreign language) and is also configured to translateterminology or jargon such that it is consumable by the user. Thesefeatures described in connection with the CHRO digital twin 8316 may bedeployed with other types of digital twins described herein, includingones for other executives, including to facilitate collaboration amongdifferent types of executives, such as for enterprise control toweractivities, such as monitoring operations, development activities, orother aspects of the enterprise across locations, departments, andfunctions. Collaboration and communication tools and associated rulesmay be configured to use company-, industry- and domain-specifictaxonomies and lexicons when representing entities, states and flowswithin the CHRO digital twin 8316, such as ones relating to health andsafety of workers, ones related to education and training, ones relatedto performance indicators, ones related to worker attributes (includingpsychographic, demographic and similar factors), and many others.

In embodiments, a CHRO digital twin 8316 may be configured to identify,interview, select, hire, and onboard new employees. In some of theseembodiments, the CHRO digital twin 8316 may be configured to research,track, and report on applicant data, including, but not limited to,employee referral data, applicant education data, applicant testingdata, applicant experience data, applicant reference data, applicantscreening data, applicant background check data, applicant interviewdata, job application data, applicant resume data, applicant coverletters, applicant offer data, and the like. The CHRO digital twin 8316may interact with and share such applicant data and reporting with otherexecutive digital twins, as described herein. The CHRO digital twin 8316may include machine learning, AI, and/or other intelligence such asanalytics, to process job applications, resumes, cover letters,applicant reference materials, applicant screening data, applicantinterview data, and the like in order to identify and select potentialnew employees and/or to identify other executives or enterprisestakeholders that may be interested in such information.

In embodiments, the EMP 8000 may obtain HR-relevant data from theenterprise's human resources management software (e.g., via an API),human capital software, workforce management software, payroll software,applicant tracking software, accounting software, employee applicantsoftware, publicly disclosed financial statements, third-party reports,tax filings, social media software, job listing websites, recruitmentsoftware, and the like.

In embodiments, a CHRO digital twin 8316 may provide an interface for anHR executive to perform one or more HR-related workflows. For example,the CHRO digital twin 8316 may provide an interface for an HR-executiveto perform, supervise, or monitor workflows, the entities involved inthe workflows, and attributes thereof, such as onboarding workflows,offboarding workflows, dismissal workflows, decision documentationworkflows, succession planning workflows, candidate assessmentworkflows, candidate screening workflows, compliance workflows,disciplinary workflows, review workflows, interview workflows, offerworkflows, employee training workflows, and many others.

In embodiments, a CHRO digital twin 8316 may leverage an executive agent8364 that is trained on a user's (e.g., an HR executive's) actions(e.g., behaviors, responses, interactions and preferences) using theexpert agent system 8008 in response to events and situationsencountered by the user (e.g., alerts, notifications, escalations,delegations, presentations of data, events, and the like). In some ofthese embodiments, the client application 8052 hosting the CHRO digitaltwin 8316 may report actions taken by the user in response to variousevents encountered by the user via the CHRO digital twin 8316. Forexample, the client application 8052 may identify events such as arequest to authorize a new hire, a request to terminate an employee, ora notification indicating that employee turnover has reached a criticalthreshold. In this example, the client application 8052 may record andreport the actions taken by the user in response to such events and mayreport the actions in relation to the identified events to the expertagent system 8008, as well as any other features that are relevant tothe event. In response, the expert agent system 8008 may train anexecutive agent 8364 on behalf of the user, such that the executiveagent may perform or recommend actions to the user when similar eventsare encountered in the future.

References to features and functions of the EMP and digital twins inthis example of a human resources department and a CHRO digital twin8316 should be understood to apply to other departments and digitaltwins, and their respective projects and workflows, except where contextindicates otherwise.

In embodiments, the executive digital twins may link to, interact with,integrate with and/or be used by a number of different applications. Forexample, the executive digital twins may be used in automatedAI-reporting tools 8360, collaboration tools 8362, in connection withexecutive agents 8364, in board meeting tools 8366, for training modules8368, and for planning tools 8370.

In embodiments, AI reporting tools 8360 assist users to report one ormore states to another user. For example, a subordinate may need toreport an identified issue to a higher-ranking member of the enterprise(e.g., CTO may wish to report an issue that needs to be addressed to theCEO). In embodiments, the AI reporting tool 8360 may be configured toreceive a request to report a state from a client device 8050. Inembodiments, the AI-reporting tool 8360 may identify the appropriaterecipients of the reported state based on the type of request, the roleof the user that issued the request and the organizational structure ofthe entity. In some embodiments, the AI-reporting tool may determine therole of the user and the recipients of the report from theorganizational digital twin of the enterprise. In some embodiments, theAI-reporting tool 8360 may determine whether the intended recipients ofa notification have access rights to the data being shared from theexecutive digital twin. For example, if the CFO is reporting to the CEO,it is likely that the CEO has access to all the enterprise's data andwill not be precluded from receiving the report. Conversely, if the CFOwishes to delegate the handling of an issue via the AI-reporting tool toan employee in her business unit, the recipient may not have access tosuch data. In this scenario, the AI-reporting tool 8360 may notify therequesting user (e.g., the CFO) that certain types of data may not beshared with the subordinate employee and may determine a manner by whichthe issue may be reported to the subordinate without sharing thenon-accessible data. Upon determining that a user has access rights toview a particular state of data, the AI-reporting tool 8360 may generatea report that is for the intended recipient. In embodiments, theAI-reporting tool may leverage the NLP services of the intelligencesystem to generate the report. In some embodiments, the AI-reportingtool 8360 may leverage an executive agent 8364 to determine when toreport a state and the appropriate recipients of the reported state. Inthese embodiments, the executive agent 8364 may be trained oninteractions of the user with the client application 8052 and digitaltwins that were previously presented to the user.

In some embodiments, the AI-reporting tool 8360 may be configured tomonitor one or more user-defined key performance indicators (KPIs).Examples of KPIs of an enterprise may include, but are not limited to,with respect to systems, facilities, processes, functions, or workforceunits: uptime (e.g., of an assembly line or other manufacturing system),capacity utilization, on-standard operating efficiency, overalleffectiveness, downtime, amount of unscheduled downtime, setup time, anamount of inventory turns, inventory accuracy, quality metrics relatingto products and services, first-pass yield amounts for the enterprise,an amount of rework required, days-sales-outstanding (DSOs), an amountof scrap or waste produced, throughput, changeover, maintenancepercentage, yield per system or unit, overall yield, industry reviews,industry ratings, customer reviews, customer ratings, editorial reviews,awards, social media and website attention metrics, search engineperformance metrics, safety metrics, health metrics, environmentalimpact metrics, political metrics, certification and testing metrics,regulatory metrics, social impact metrics, financial and investmentmetrics, corporate bond ratings, trade association metrics, unionmetrics, lobbying organization ratings, advertising performance metrics,referral metrics, and many others. Additional or alternative KPI metricsmay be defined by a user. Examples of these KPI metrics may include anamount or percentage of failed audits, a number or percentage ofdeliveries that are on-time/late, a number of customer returns, a numberof employee training hours, employee turnover percentage, number ofreportable health or safety incidents, revenue per employee, profit peremployee, schedule attainment metrics, total cycle time, and the like.

In embodiments, the collaboration tools 8362 include various tools thatallow collaboration between executives of the enterprise. Inembodiments, the collaboration tools include digital-twin enabled videoconferencing. In these embodiments, the EMP 8000 may presentparticipants in the video conference with the requested view of anenterprise digital twin. For example, during a Board meeting, a CTOproposing an update to the machinery or equipment in a facility maypresent an environment digital twin of the facility where the updates tothe machinery or equipment would be made. In this example, the CTO mayillustrate the results of simulations performed in the facility withoutthe updates and with the updates. The simulation may illustrate how theupdate may benefit the enterprise using a number of selected metrics(e.g., throughput, profits, employee safety, or the like). Collaborationand communication tools and associated rules may be configured to usecompany-, industry- and domain-specific taxonomies and lexicons whenrepresenting entities, states and flows within the digital twin.

In embodiments, executive agents 8364 are expert agents that are trainedto perform tasks on behalf of executive users. As discussed, in someembodiments, a client application may monitor the user of the clientapplication by a user when using the client application 8052. In theseembodiments, the client application 8052 may monitor the states of anexecutive digital twin that the user drills down into, the states thatthe user reports to a superior and/or delegates to a team member in herrespective business unit, decisions that are made, and the like. As theuser uses the client application 8052, the expert agent system 8008 maytrain one or more machine-learned models on behalf of the particularuser, such that the models may be leveraged by an executive agent 8364to perform tasks on behalf of or recommend actions to the user.

In embodiments, Board meeting tools 8366 are tools that are used toprepare for, to access within and/or to follow-up on board and similarmeetings, such as Board of Directors, Board of Trustees, shareholdermeetings, annual meetings, investor meetings, and other importantmeetings. References to Board meetings herein should be understood toencompass these and other important meetings that require executivepreparation, attendance and/or attention. In embodiments, Board meetingtools 8366 may allow different users to present one or more states of anenterprise digital twins within the context of a Board report or Boardmeeting. For example, a user (e.g., a COO) may share a simulation of aproposed logistics solution from the COO digital twin 8366 with one ormore devices (e.g., a device in the Board room and/or devices ofparticipants accessing the Board meeting remotely). In embodiments, aBoard meeting tool 8366 may limit access to certain types of data basedon time, scope, and permissions. For example, a Board meeting tool 8366may require that all geolocations that board members be registeredbefore a Board meeting (e.g., Board room, designated home offices forthose joining by phone or video, and the like), such that some or all ofthe data depicted in a digital twin that is being presented can only beviewed on a device that is at one of the registered geolocations and/oronly for a defined duration, such as from a few hours before through afew hours after a meeting, or only during the meeting. Similarly, inembodiments, the Board meeting tools 8366 may limit access to some orall of the data shared in a presented digital twin to particular times(e.g., during the Board meeting or the day of the Board meeting). Otherexamples of board meeting tools 8366 are discussed throughout theapplication.

In embodiments, training modules 8368 may include software tools thatare used to train a user. In embodiments, the training modules 8368 mayleverage digital twins to improve executive training for an enterprise.For example, a training module 8368 may provide real-world examples thatare based on the data collected from the enterprise. The training module8368 may present the user with different scenarios via an executivedigital twin 8368 and the user may take actions. Based on the actions,the training module 8368 may request a simulation from the EMP 8000,which in turn returns the results to the user. In this way, the user maybe trained on scenarios that are based on the actual enterprise of theuser.

In embodiments, planning tools 8370 are software tools that leveragedigital twins to assist users to make plans for the enterprise. Inembodiments, a planning tool 8370 may be configured to provide agraphical user interface that allows an executive to make plans (e.g.,budgets, defining KPIs, etc.). In some embodiments, the planning tool8370 may be configured to request a simulation from the IMP 8000 giventhe parameters set in the created plan. In response, the EMP 8000 mayreturn the results of the simulation and the user can determine whetherto adjust the plan. In this way, the user may iteratively refine theplan to achieve one or more objectives. In embodiments, an executiveagent 8362 may monitor the track the actions taken while the plan isbeing refined by the user so that the expert agent system 8008 may trainthe executive agent 8362 to generate or recommend plans to the user inthe future.

The enterprise digital twins may be leveraged and/or interface withother software applications without departing from the scope of thedisclosure.

FIG. 84 illustrates an example implementation of the EMP 8000. In thisexample, the EMP 8000 is in communication with a plurality of clientapplications 8052 and a set of enterprise assets 8400. In the example,the EMP 8000 receives enterprise data from a set of enterprise entities8400, such as the sensor system 8022, physical entities 8402, digitalentities 8404, computational entities 8406, and/or network entities 8408belonging to and/or associated with the enterprise. In embodiments, theenterprise data may relate to environments, processes, and/or acondition of the enterprise. For example, the sensor system 8022 may bedeployed within an enterprise facility (e.g., manufacturing facility,warehouse, distribution center, logistics facility, transportationfacility, office building, customer location, retail location,agricultural facility, natural resource extraction facility, or thelike) of the enterprise, whereby the sensor system 8022 provides sensorreadings (e.g., vibration data, location data, motion data, temperaturedata, pressure data, or the like) relating to the facility in general ora piece of machinery, equipment, or other physical or workforce assetwithin the facility. Within the facility, a number of physical assets(e.g., robots, autonomous vehicles, smart equipment, personnel and thelike) or other entities may output data streams relating to theoperation of the assets or other entities. Additionally oralternatively, the enterprise may include a number of digital assets(e.g., CRM, ERP, databases, or the like) that provide data streamsrelating to sales, costs, human resources or the like. The networkentities may provide networking-related data, including bandwidth, APIrequests, throughput, detected cyber-attacks, or the like. Thecomputational entities may provide data relating to a computinginfrastructure of an enterprise. In some embodiments, the enterprisemanagement system 8000 may receive data from other sources as well,including third-party data 8038 from third-party data providers. Takenin combination, the data from the enterprise assets 8400 and/or otherdata sources may provide information relating to the status of theindustrial facility and the machinery contained therein, the state ofvarious processes (e.g., industrial processes, sales workflows, hiringprocesses, logistics workflows, and the like), the efficiencies of theprocesses, the financial health of the enterprise, and the like.

In embodiments, the enterprise entities may communicate directly withthe EMP 8000 via a communication network. Additionally or alternatively,one or more of the enterprise assets may stream data to a local datacollection system 8420 that collects and stores enterprise data locally.In some embodiments, the local data collection system 8420 may providethe collected data to an edge intelligence system 8422 of theenterprise.

In embodiments, the edge intelligence system 8422 may be executed by anedge device 8042 configured to receive data, such as from the local datacollection systems 8420, a local sensor system 8022, or other enterpriseentities 8400 that are located in or near a physical location of theentities (e.g., at an industrial facility) and may perform one or moreedge-related processes relating to the received data. The edge devicemay be a pre-configured and/or substantially self- or automaticallyconfiguring computing device, such as an “edge intelligence in a box”device. An edge-related process may refer to a process that is performedat an edge device in order to store sensor data, reduce bandwidth on acommunication network, and/or reduce the computational resourcesrequired at a backend system. Examples of edge processes can includedata filtering, signal filtering, data processing, compression,encoding, quick-predictions, quick-notifications, emergency alarming,and the like, and may include creation of automated smart data bands.For example, the edge intelligence system 8422 may determine whether totransmit a subset of the data to the EMP 8000 or to store the subset ofthe data locally until it is explicitly requested from the EMP 8000. Inanother example, the edge intelligence system 8422 may be configured tocompress data streams (e.g., sensor data streams) to improve datathroughput of high-volume data streams (e.g., vibration data). In someembodiments, the edge intelligence system 8422 may be configured toanalyze the high-volume data to determine whether to compress or streama raw data stream. In some embodiments, the local data collection system8420 and the edge intelligence system 8422 may be embodied in edgedevices 8042 of the enterprise. In some embodiments, the edgeintelligence system 8422 may communicate data to the EMP 8000. In someof these embodiments, the edge intelligence system 8422 communicatesdata to the EMP 8000 via a network enhancement system 8424.

In embodiments, the network enhancement system 8424 may be configured tooptimize flow of data transmitted from one or both of the edgeintelligence system 8422 and the local data collection system 8420 andreceived by the EMP 8000. For example, a local data collection system8420 may be configured to collect data from one or more real worldenvironments, entities, ecosystems, and/or processes, which may beanalyzed by a connected edge intelligence system 8422. In this example,the edge intelligence system 8422 may transmit the collected data to thenetwork enhancement system 8424, which may optimize transmission of thedata to the EMP 8000 for processing and implementation by the EMP 8000.The EMP 8000 may store, analyze, or otherwise process the transmitteddata to the client applications 8052, such that the client applications8052 may update enterprise digital twins (e.g., role-based digitaltwins, environment digital twins, cohort digital twins, and the like)that are hosted by the client applications 8052.

In embodiments, the network enhancement system 8424 may include one ormore signal amplifiers, signal repeaters, digital filters, analogfilters, digital-to-analog converters, analog-to-digital converterand/or antennae configured to optimize the flow of data. In someembodiments, the network enhancement system may include a wirelessrepeater system such as is disclosed by U.S. Pat. No. 7,623,826 toPergal, the entirety of which is hereby incorporated by reference. Thenetwork enhancement system 8424 may optimize the flow of data by, forexample, filtering data, repeating data transmission, amplifying datatransmission, adjusting one or more sampling rates and/or transmissionrates, and implementing one or more data communication protocols.

In embodiments, the network enhancement system 8424 may include one ormore processors configured to perform digital signal processing tooptimize the flow of data. The one or more processors may implementoptimization algorithms to optimize the flow of data. The one or moreprocessors may determine one or more optimal paths in a network, thenetwork enhancement system 8424 transmitting the data along the one ormore optimal paths. The network enhancement system 8424 may beconfigured to implement a software filter via the one or moreprocessors. The software filter may filter data before transmission tothe EMP 8000, for example to lower network bandwidth consumed by datatransmission. The one or more processors may determine that portions ofdata are relevant only to one or more intended recipients, such asdigital twins, executive agents, collaboration suites, or othercomponents of the EMP 8000 and determine optimal paths based uponintended recipients of the portions of data.

In embodiments, the network enhancement system 8424 may be configured tooptimize data flow between a plurality of nodes over a plurality of datapaths. In some embodiments, the network enhancement system 8424 maytransmit a first portion of data over a first path of the plurality ofdata paths and a second portion of data over a second path of theplurality of data paths. The network enhancement system 8424 maydetermine that one or more data paths, such as the first data path, thesecond data path, other data paths, are advantageous for transmission ofone or more portions of data. The network enhancement system 8424 maymake determinations of advantageous data paths based upon one or morenetworking variables, such as one or more types of data beingtransmitted, one or more protocols being suitable for transmission,present and/or anticipated network congestion, timing of datatransmission, present and/or anticipated volumes of data being or to betransmitted, and the like. Protocols suitable for transmission mayinclude transmission control protocol (TCP), user datagram protocol(UDP), and the like. In some embodiments, the network enhancement systemmay be configured to implement a method for data communication such asis disclosed by U.S. Pat. No. 9,979,664 to Ho et al., the entirety ofwhich is hereby incorporated by reference.

The EMP 8000 receives enterprise data (e.g., directly or via the networkenhancement system 8424, an edge intelligence system 8422, a local datacollection system 8420 or from any other data source). In embodiments,the digital twin system 8004 may structure and/or store the enterprisedata in one or more digital twin databases (e.g., graph databases,relational databases, SQL databases, distributed databases, blockchains,caches, servers, and/or the like). In embodiments, the clientapplication 8052 requests an enterprise digital twin 8410 from the EMP8000. In response, the digital twin system 8004 may generate and servethe requested enterprise digital twin 8410 (e.g., a role-based digitaltwin, executive digital twin, environment digital twin, process digitaltwin, cohort digital twins, or the like) to the client application 8052,whereby the enterprise digital twin 8410 may include the enterprise dataand/or data that was derived from the enterprise data (e.g., by theintelligence services system). The client application 8052 may providean interface for the user of the client application 8052 to interactwith the requested digital twin 8410. For example, the user may delegatetasks relating to a depicted state to subordinates and/or may notify asuperior of a depicted state via the digital twin interface. In anotherexample, the user may drill down into a particular state and mayinitiate a corrective action via the digital twin interface. In someembodiments, the client application 8052 may allow the user to share thedigital twin 8410 (or a portion thereof) within a collaboration tool8414 or access collaboration features of a collaboration tool 8414within the twin 8410. For example, the client application 8052 may allowthe user to share a depicted state of the digital twin 8410 into a boardmeeting collaboration tool Additionally or alternatively, an expertagent 8364 may monitor the interactions of the user with the digitaltwin and may report the interactions to the expert agent system 8008 ofthe EMP. In embodiments, the expert agent system 8008 may receive theinteractions and may train the expert agent 8364 based on theinteractions with the digital twin, as well as outcomes stemming fromthe expert agent. For example, the expert agent may be trained toidentify situations where the user delegates tasks or notifies asuperior.

The executive digital twins discussed with respect to FIG. 71 areprovided for example and not intended to limit the scope of thedisclosure. Additional and/or alternative data types may be included ina respective type of executive digital twin.

FIG. 73 illustrates an example method 8510 for configuring and servingan enterprise digital twin. In embodiments, the method may be executedby the digital twin system 8004. The method may be performed withrespect to different types of enterprise digital twins, includingrole-based digital twins (e.g., executive digital twins), cohort digitaltwins, environment digital twins, process digital twins, and/or thelike.

At 8512, the structural views for a particular type of digital twin areselected. In embodiments, the structural views can be stored in a graphdatabase (representing interconnected data) or in a geospatial database(representing coordinates of actual facilities).

At 8514, associated transactional data for the digital twin is selected.In embodiments, a combination of interaction data and transaction datais selected at grain that is suitable for the dynamic interaction withinthe digital twin is selected. This selection process may involve dynamicconfiguration of the structure, functions and features of a data mart orother summarization system and/or may work dynamically using typicallyhigh-performance database storage mechanisms (such as columnar databasesor in memory databases).

At 8516, embellishment and/or augmentation data for the digital twin isselected. In embodiments, embellishment data are the associatedattributes that can be tied to elements within the executive digitaltwin. For example, in generating an environment digital twin of afacility, embellishment or augmentation data may include the ages ofmachinery or other assets in the facility, the names of key third-partysuppliers that could replace items with supply chain deliveries, theinputs or outputs of process flows that occur within the facility,identities of managers, indicators of states and flows, and many others.In an abstract executive digital twin the embellishment data may includesocial media data, for example sentiment analytics that can beassociated with the customer hierarchical views.

At 8518, a representation medium for the digital twin is selected. Inembodiments, the final representation can be multi-faceted, this caninclude a range of devices from simple mobile phone-based devices andtouchscreen tablets to special-purpose devices and/or immersive AR/VRheadsets, among many others. The representation medium impacts thevolume and nature of data that is preferably selected in the earliersteps. In embodiments, selection of a representation medium is providedas a feedback indicator to the data and networking pipeline, such thatfiltering and data path selection can be undertaken with awareness ofend device and other capabilities and requirements of the representationmedium. This may occur automatically, such as by an agent that istrained to provide context-sensitive feedback based on a training set ofoutcomes.

At 8520, the perspective views are constructed. In embodiments, theperspective builder 8110 generates a level and nature of data thatallows for different types of user to interact with the digital twinwhile gaining the appropriate level of perspective. For example, with aCEO-level view the CEO may require the context of third-partyalternatives, market forces, and current strategic initiatives. In thisexample, the perspective builder 8110 takes these considerations intoaccount in producing the level of digital twin appropriate for the CEO,furthermore this will impact the data selection process as differentgrains of data are appropriate for the different views. These differentperspectives can be simultaneously interacted with various rolesallowing the executive to provide their guidance on the same topic whileseeing and interaction with information relevant to their specificneeds.

At 8522, user notifications are enabled. In embodiments, notificationswithin the digital twin are controlled by the grain of the data selectedand the required perspective. For example, a CTO level view requiresnotifications of various technology changes and technology marketforces, the CTO digital twin is constantly being overlaid with thesenotifications that are structurally associated with the relevant part ofthe digital environment abstract or concrete. For example, in anorganizational chart the CTO could be seeing the implementation optionsfor new technology to provide more efficient communication betweenorganizational units in strategic planning exercise to acquire a newcompany. Simultaneously the CFO is seeing the financial impacts of thesevarious options, and the CEO is being notified of decisions that mightimpact the future market opportunities regarding the upcoming companyacquisition.

The method is provided for example only. Additional and/or alternativemethods may be performed to generate and serve digital twins withoutdeparting from the scope of the disclosure.

The method of FIG. 73 is provided for example and not intended to limitthe scope of the disclosure. The method may include additional oralternative operations.

FIG. 74 illustrates an example set of operations of a method 8600 forconfiguring an organizational digital twin. In embodiments, the methodmay be executed at least in part by the digital twin system 8004. It isappreciated that the method may be executed by other suitable computingsystems without departing from the scope of the disclosure.

At 8610 an organizational chart of an enterprise is determined. Inembodiments, a user may upload the organizational chart via a GUIdisplayed to the user. In some embodiments, the digital twin system 8004or a connected component may crawl one or more websites (e.g., theenterprise website, a social networking website, or the like) and mayparse the crawled website(s) to determine the organizational chart.

At 8612, the organizational framework of the enterprise is updated basedon user input. In embodiments, a user may define roles within theenterprise to individuals listed in the organizational chart, grantaccess rights to different roles and/or individuals, grant permissionsto individuals and/or roles, and may define relationships between rolesand/or individuals. In embodiments, the relationships may representreporting structures, teams, business units, and the like.

At 8614, an organizational digital twin of the enterprise is generatedand deployed. In embodiments, the digital twin system 8004 may generatethe organizational digital twin by connecting data from the enterpriseto the organizational chart. This may include information relating tothe individuals, such as birthdate, social security or tax id, role,relationships, citizenship, employment status, salary, stock holdings,title, current status, goals or targets, and the like. Once deployed,the organizational chart may be continuously updated from one or moreenterprise data sources. In embodiments, the organizational digital twinmay be leveraged to determine the roles of individuals within anorganization and/or the reporting structure of the digital twin.

The method of FIG. 74 is provided for example and not intended to limitthe scope of the disclosure. The method may include additional oralternative operations.

FIG. 75 illustrates an example set of operations of a method 8700 forgenerating an executive digital twin. In embodiments, the method may beexecuted at least in part by the digital twin system 8004. It isappreciated that the method may be executed by other suitable computingsystems without departing from the scope of the disclosure.

At 8710, a request for an executive digital twin is received from auser. In embodiments, the digital twin system 8004 may receive a requestfor an executive digital twin from a user device associated with a user,such as a mobile device, a personal computer, a VR device, or the like.The request may indicate an identity of the user and/or a role of theuser.

At 8712, a role of the user is determined. In embodiments, the digitaltwin system 8004 may determine a role of the user from the requestand/or from an organizational digital twin of an enterprise associatedwith the user. In embodiments, the organizational digital twin mayindicate the role of the user, the permissions of the user, the accessrights of the user, restrictions of the user, and a reporting structureof the user.

At 8714, a configuration of the executive digital twin is determinedbased on the role of the user. In embodiments, the configuration of theexecutive digital twin indicates a set of states that re to be depictedin the executive digital twin and a granularity of the digital twin. Inembodiments, the configuration of the executive digital twin is storedin a configuration file in the digital twin data store associated withthe enterprise. The configuration file may define the initial states ofthe digital twin and the granularities of the states.

At 8716, a digital twin is generated based on one or more data sourcescorresponding to the enterprise. In embodiments, the digital twin system8004 may determine the appropriate perspective for the requested digitaltwin based on the configuration of the digital twin and any accessrights or restrictions of the user. In embodiments, the restrictions mayinclude data restrictions, interaction restrictions, depth of datarestrictions, usage restrictions, length of visibility restrictions,that the user may have. In some embodiments, generating the requesteddigital twin may include identifying the appropriate data sources forthe digital twin given the perspective and obtaining any data thatinitially parameterizes the executive digital twin from the datasources.

At 8718, the executive digital twin is served to a user device of theuser. In embodiments, the digital twin system 8004 may provide a file(e.g., a JSON file) containing the executive digital twin data and anydata structures or visual elements that are needed to depict theexecutive digital twin by the user device. In embodiments, the digitaltwin system 8004 may also stream one or more real-time data or near-realtime data streams to the user device (e.g., via a data bus), such thatthe executive digital twin may be updated with fresh data as the userinteracts with the executive digital twin. The user may then interactwith the digital twin. For example, the user may delegate tasks via theexecutive digital twin, request simulations via the executive digitaltwin, drill down into or zoom out of states depicted in the executivedigital twin, report states to a supervisor via the executive digitaltwin, and/or the like.

The method of FIG. 75 is provided for example and not intended to limitthe scope of the disclosure. The method may include additional oralternative operations.

Artificial Intelligence and Neural Network Embodiments

Referring to FIGS. 76 through 103 , in embodiments of the presentdisclosure, including ones involving artificial intelligence 1160,expert systems, self-organization, machine learning, automation(including robotic process automation, remote control, autonomousoperation, automated configuration, and the like), adaptive intelligenceand adaptive intelligent systems, prediction, classification,optimization, and the like, may benefit from the use of a neural networkor other artificial intelligence system, such as a neural net trainedfor pattern recognition, for classification of one or more parameters,characteristics, or phenomena, for support of autonomous control, andother purposes. References to artificial intelligence, neural network orneural net throughout this disclosure should be understood to encompassa wide range of different types of neural networks, machine learningsystems, artificial intelligence systems, and the like, such as feedforward neural networks, radial basis function neural networks,self-organizing neural networks (e.g., Kohonen self-organizing neuralnetworks), recurrent neural networks, modular neural networks,artificial neural networks, physical neural networks, multi-layeredneural networks, convolutional neural networks, hybrids of neuralnetworks with other expert systems (e.g., hybrid fuzzy logic—neuralnetwork systems), Autoencoder neural networks, probabilistic neuralnetworks, time delay neural networks, convolutional neural networks,regulatory feedback neural networks, radial basis function neuralnetworks, recurrent neural networks, Hopfield neural networks, Boltzmannmachine neural networks, self-organizing map (SOM) neural networks,learning vector quantization (LVQ) neural networks, fully recurrentneural networks, simple recurrent neural networks, echo state neuralnetworks, long short-term memory neural networks, bi-directional neuralnetworks, hierarchical neural networks, stochastic neural networks,genetic scale RNN neural networks, committee of machines neuralnetworks, associative neural networks, physical neural networks,instantaneously trained neural networks, spiking neural networks,neocognition neural networks, dynamic neural networks, cascading neuralnetworks, neuro-fuzzy neural networks, compositional pattern-producingneural networks, memory neural networks, hierarchical temporal memoryneural networks, deep feed forward neural networks, gated recurrent unit(GCU) neural networks, auto encoder neural networks, variational autoencoder neural networks, de-noising auto encoder neural networks, sparseauto-encoder neural networks, Markov chain neural networks, restrictedBoltzmann machine neural networks, deep belief neural networks, deepconvolutional neural networks, de-convolutional neural networks, deepconvolutional inverse graphics neural networks, generative adversarialneural networks, liquid state machine neural networks, extreme learningmachine neural networks, echo state neural networks, deep residualneural networks, support vector machine neural networks, neural Turingmachine neural networks, and/or holographic associative memory neuralnetworks, or hybrids or combinations of the foregoing, or combinationswith other expert systems, such as rule-based systems, model-basedsystems (including ones based on physical models, statistical models,flow-based models, biological models, biomimetic models, and the like).

The foregoing neural networks may have a variety of nodes or neurons,which may perform a variety of functions on inputs, such as inputsreceived from sensors or other data sources, including other nodes.Functions may involve weights, features, feature vectors, and the like.Neurons may include perceptron, neurons that mimic biological functions(such as of the human senses of touch, vision, taste, hearing, andsmell), and the like. Continuous neurons, such as with sigmoidalactivation, may be used in the context of various forms of neural net,such as where back propagation is involved.

In many embodiments, an expert system or neural network may be trained,such as by a human operator or supervisor, or based on a data set,model, or the like. Training may include presenting the neural networkwith one or more training data sets that represent values (including themany types described throughout this disclosure), as well as one or moreindicators of an outcome, such as an outcome of a process, an outcome ofa calculation, an outcome of an event, an outcome of an activity, or thelike. Training may include training in optimization, such as training aneural network to optimize one or more systems based on one or moreoptimization approaches, such as Bayesian approaches, parametric Bayesclassifier approaches, k-nearest-neighbor classifier approaches,iterative approaches, interpolation approaches, Pareto optimizationapproaches, algorithmic approaches, and the like. Feedback may beprovided in a process of variation and selection, such as with a geneticalgorithm that evolves one or more solutions based on feedback through aseries of rounds.

In embodiments, a plurality of neural networks may be deployed in acloud platform that receives data streams and other inputs collected(such as by mobile data collectors) in one or more environments andtransmitted to the cloud platform over one or more networks, includingusing network coding to provide efficient transmission. In the cloudplatform, optionally using massively parallel computational capability,a plurality of different neural networks of various types (includingmodular forms, structure-adaptive forms, hybrids, and the like) may beused to undertake prediction, classification, control functions, andprovide other outputs as described in connection with expert systemsdisclosed throughout this disclosure. The different neural networks maybe structured to compete with each other (optionally including useevolutionary algorithms, genetic algorithms, or the like), such that anappropriate type of neural network, with appropriate input sets,weights, node types and functions, and the like, may be selected, suchas by an expert system, for a specific task involved in a given context,workflow, environment process, system, or the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a feed forwardneural network, which moves information in one direction, such as from adata input, like a source of data about an individual, through a seriesof neurons or nodes, to an output. Data may move from the input nodes tothe output nodes, optionally passing through one or more hidden nodes,without loops. In embodiments, feed forward neural networks may beconstructed with various types of units, such as binary McCulloch-Pittsneurons, the simplest of which is a perceptron.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a radial basisfunction (RBF) neural network, which may be preferred in some situationsinvolving interpolation in a multi-dimensional space (such as whereinterpolation is helpful in optimizing a multi-dimensional function,such as for optimizing a data marketplace as described here, optimizingthe efficiency or output of a power generation system, a factory system,or the like, or other situation involving multiple dimensions. Inembodiments, each neuron in the RBF neural network stores an examplefrom a training set as a “prototype.” Linearity involved in thefunctioning of this neural network offers RBF the advantage of nottypically suffering from problems with local minima or maxima.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a radial basisfunction (RBF) neural network, such as one that employs a distancecriterion with respect to a center (e.g., a Gaussian function). A radialbasis function may be applied as a replacement for a hidden layer, suchas a sigmoidal hidden layer transfer, in a multi-layer perceptron. AnRBF network may have two layers, such as where an input is mapped ontoeach RBF in a hidden layer. In embodiments, an output layer may comprisea linear combination of hidden layer values representing, for example, amean predicted output. The output layer value may provide an output thatmay be the same as or similar to that of a regression model instatistics. In classification problems, the output layer may be asigmoid function of a linear combination of hidden layer values,representing a posterior probability. Performance in both cases may beoften improved by shrinkage techniques, such as ridge regression inclassical statistics. This corresponds to a prior belief in smallparameter values (and therefore smooth output functions) in a Bayesianframework. RBF networks may avoid local minima, because the onlyparameters that are adjusted in the learning process are the linearmapping from hidden layer to output layer. Linearity ensures that theerror surface may be quadratic and therefore has a single minimum. Inregression problems, this can be found in one matrix operation. Inclassification problems, the fixed non-linearity introduced by thesigmoid output function may be handled using an iteratively. Re-weightedleast squares function or the like.

In embodiments, RBF networks may use kernel methods such as supportvector machines (SVM) and Gaussian processes (where the RBF may be thekernel function). A non-linear kernel function may be used to projectthe input data into a space where the learning problem can be solvedusing a linear model.

In embodiments, an RBF neural network may include an input layer, ahidden layer and a summation layer. In the input layer, one neuronappears in the input layer for each predictor variable. In the case ofcategorical variables, N−1 neurons are used, where N is the number ofcategories. The input neurons may, in embodiments, standardize the valueranges by subtracting the median and dividing by the interquartilerange. The input neurons may then feed the values to each of the neuronsin the hidden layer. In the hidden layer, a variable number of neuronsmay be used (determined by the training process). Each neuron mayconsist of a radial basis function that may be centered on a point withas many dimensions as a number of predictor variables. The spread (e.g.,radius) of the RBF function may be different for each dimension. Thecenters and spreads may be determined by training. When presented with avector of input values from the input layer, a hidden neuron may computea Euclidean distance of the test case from the neuron's center point andthen apply the RBF kernel function to this distance, such as using thespread values. The resulting value may then be passed to the summationlayer. In the summation layer, the value coming out of a neuron in thehidden layer may be multiplied by a weight associated with the neuronand may add to the weighted values of other neurons. This sum becomesthe output. For classification problems, one output may be produced(with a separate set of weights and summation units) for each targetcategory. The value output for a category is the probability that thecase being evaluated has that category. In training of an RBF, variousparameters may be determined, such as the number of neurons in a hiddenlayer, the coordinates of the center of each hidden-layer function, thespread of each function in each dimension, and the weights applied tooutputs as they pass to the summation layer. Training may be used byclustering algorithms (such as k-means clustering), by evolutionaryapproaches, and the like.

In embodiments, a recurrent neural network may have a time-varying,real-valued (more than just zero or one) activation (output). Eachconnection may have a modifiable real-valued weight. Some of the nodesare called labeled nodes, some output nodes, and other hidden nodes. Forsupervised learning in discrete time settings, training sequences ofreal-valued input vectors may become sequences of activations of theinput nodes, one input vector at a time. At each time step, eachnon-input unit may compute its current activation as a nonlinearfunction of the weighted sum of the activations of all units from whichit receives connections. The system can explicitly activate (independentof incoming signals) some output units at certain time steps.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a self-organizingneural network, such as a Kohonen self-organizing neural network, suchas for visualization of views of data, such as low-dimensional views ofhigh-dimensional data. The self-organizing neural network may applycompetitive learning to a set of input data, such as from one or moresensors or other data inputs from or associated with an individual. Inembodiments, the self-organizing neural network may be used to identifystructures in data, such as unlabeled data, such as in data from variousunstructured sources, such as social media sources about an individual,where sources of the data are unknown (such as where data comes fromvarious unknown or uncertain sources). The self-organizing neuralnetwork may organize structures or patterns in the data, such that theycan be recognized, analyzed, and labeled, such as identifying structuresas corresponding to individuals, disease conditions, health states,activity states, and the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a recurrent neuralnetwork, which may allow for a bi-directional flow of data, such aswhere connected units (e.g., neurons or nodes) form a directed cycle.Such a network may be used to model or exhibit dynamic temporalbehavior, such as involved in dynamic systems, such as a wide variety ofthe disease conditions, health states, and biological systems describedthroughout this disclosure, such as a body experiencing multipledifferent diseases or health conditions, or the like, where dynamicsystem behavior involves complex interactions that an observer maydesire to understand, diagnose, predict, control, treat and/or optimize.For example, the recurrent neural network may be used to anticipate thestate (such as a maintenance state, a health state, a disease state, orthe like), of an individual, such as one interacting with a system,performing an action, or the like. In embodiments, the recurrent neuralnetwork may use internal memory to process a sequence of inputs, such asfrom other nodes and/or from sensors and other data inputs from anenvironment, of the various types described herein, such as a socialnetwork, a home or work environment, a health care environment, arecreational or sports environment, or the like. In embodiments, therecurrent neural network may also be used for pattern recognition, suchas for recognizing a person based on a biomarker, a face, a voice orsound signature, a heat signature, a set of feature vectors in an image,a chemical signature, or the like. In a non-limiting example, arecurrent neural network may recognize a change or shift in a state of ahuman by learning to classify the shift or change from a training dataset consisting of a stream of data from unstructured data sources, suchas social media sources.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a modular neuralnetwork, which may comprise a series of independent neural networks(such as ones of various types described herein) that are moderated byan intermediary. Each of the independent neural networks in the modularneural network may work with separate inputs, accomplishing subtasksthat make up the task the modular network as a whole is intended toperform. For example, a modular neural network may comprise a recurrentneural network for pattern recognition, such as to recognize what typeof person, condition, state, or the like is being sensed by one or moresensors that are provided as input channels to the modular network andan RBF neural network for optimizing a system, protocol, or the like,once understood. The intermediary may accept inputs of each of theindividual neural networks, process them, and create output for themodular neural network, such an appropriate control parameter, aprediction of state, or the like.

Combinations among any of the pairs, triplets, or larger combinations,of the various neural network types described herein, are encompassed bythe present disclosure. This may include combinations where an expertsystem uses one neural network for recognizing a pattern (e.g., apattern indicating a problem or fault condition) and a different neuralnetwork for self-organizing an activity or work flow based on therecognized pattern (such as providing an output governing autonomouscontrol of a system in response to the recognized condition or pattern).This may also include combinations where an expert system uses oneneural network for classifying an item (e.g., identifying a machine, acomponent, or an operational mode) and a different neural network forpredicting a state of the item (e.g., a fault state, an operationalstate, an anticipated state, a maintenance state, a. predicted state, orthe like). Modular neural networks may also include situations where anexpert system uses one neural network for determining a state or context(such as a state of a machine, a process, a work flow, a storage system,a network, a data collector, or the like) and a different neural networkfor self-organizing a process involving the state or context (e.g., adata storage process, a network coding process, a network selectionprocess, a data processing process, or other process described herein).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a physical neuralnetwork where one or more hardware elements may be used to perform orsimulate neural behavior. One or more hardware nodes may be configuredto stream output data resulting from the activity of the neural net.Hardware nodes, which may comprise one or more chips, microprocessors,integrated circuits, programmable logic controllers,application-specific integrated circuits, field-programmable gatearrays, or the like, may be provided to optimize the speed, input/outputefficiency, energy efficiency, signal to noise ratio, or other parameterof some part of a neural net of any of the types described herein.Hardware nodes may include hardware for acceleration of calculations(such as dedicated processors for performing basic or more sophisticatedcalculations on input data to provide outputs, dedicated processors forfiltering or compressing data, dedicated processors for decompressingdata, dedicated processors for compression of specific file or datatypes (e.g., for handling image data, video streams, acoustic signals,vibration data, thermal images, heat maps, or the like), and the like. Aphysical neural network may be embodied in a data collector, edgeintelligence system, adaptive intelligent system, mobile data collector,IoT monitoring system, or other system described herein, including onethat may be reconfigured by switching or routing inputs in varyingconfigurations, such as to provide different neural net configurationswithin the system for handling different types of inputs (with theswitching and configuration optionally under control of an expertsystem, which may include a software-based neural net located on thedata collector or remotely). A physical, or at least partially physical,neural network may include physical hardware nodes located in a storagesystem, such as for storing data within machine, a product, or the like,such as for accelerating input/output functions to one or more storageelements that supply data to or take data from the neural net. Aphysical, or at least partially physical, neural network may includephysical hardware nodes located in a network, such as for transmittingdata within, to or from an environment, such as for acceleratinginput/output functions to one or more network nodes in the net,accelerating relay functions, or the like. In embodiments, of a physicalneural network, an electrically adjustable resistance material may beused for emulating the function of a neural synapse. In embodiments, thephysical hardware emulates the neurons, and software emulates the neuralnetwork between the neurons. In embodiments, neural networks complementconventional algorithmic computers. They may be trained to performappropriate functions without the need for any instructions, such asclassification functions, optimization functions, pattern recognitionfunctions, control functions, selection functions, evolution functions,and others.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a multilayeredfeed forward neural network, such as for complex pattern classificationof one or more items, phenomena, modes, states, or the like. Inembodiments, a multilayered feed forward neural network may be trainedby an optimization technical, such as a genetic algorithm, such as toexplore a large and complex space of options to find an optimum, ornear-optimum, global solution. For example, one or more geneticalgorithms may be used to train a multilayered feed forward neuralnetwork to classify complex phenomena, such as to recognize complexoperational modes or states of individuals, such as modes involvingcomplex interactions among entities (including interference effects,amplifying effects, and the like), modes involving non-linear phenomena,such as impacts of interaction of protocols, which may make analysis ofsymptoms or diagnosis of conditions of entities difficult, modesinvolving critical risks, such as where multiple, simultaneousconditions occur, making root cause analysis difficult, and others. Inembodiments, a multilayered feed forward neural network may be used toclassify results from monitoring unstructured data, such as form socialmedia.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a feed-forward,back-propagation multi-layer perceptron (MLP) neural network, such asfor handling one or more remote sensing applications, such as for takinginputs from sensors distributed throughout various human-inhabitedenvironments, including home and work environments, businessenvironments, and the like. In embodiments, the MLP neural network maybe used for classification of physical environments. This may includefuzzy classification.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use astructure-adaptive neural network, where the structure of a neuralnetwork may be adapted, such as based on a rule, a sensed condition, acontextual parameter, or the like. For example, if a neural network doesnot converge on a solution, such as classifying an item or arriving at aprediction, when acting on a set of inputs after some amount oftraining, the neural network may be modified, such as from a feedforward neural network to a recurrent neural network, such as byswitching data paths between some subset of nodes from unidirectional tobi-directional data paths. The structure adaptation may occur undercontrol of an expert system, such as to trigger adaptation uponoccurrence of a trigger, rule or event, such as recognizing occurrenceof a threshold (such as an absence of a convergence to a solution withina given amount of time) or recognizing a phenomenon as requiringdifferent or additional structure (such as recognizing that a system maybe varying dynamically or in a non-linear fashion).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an autoencoder,autoassociator or Diabolo neural network, which may be similar to amultilayer perceptron (MLP) neural network, such as where there may bean input layer, an output layer and one or more hidden layers connectingthem. However, the output layer in the auto-encoder may have the samenumber of units as the input layer, where the purpose of the MLP neuralnetwork may be to reconstruct its own inputs (rather than just emittinga target value). Therefore, the auto encoders are may operate as anunsupervised learning model. An auto encoder may be used, for example,for unsupervised learning of efficient codings, such as fordimensionality reduction, for learning generative models of data, andthe like. In embodiments, an auto-encoding neural network may be used toself-learn an efficient network coding for transmission of data from orabout an individual over one or more networks, which may include socialnetworks. In embodiments, an auto-encoding neural network may be used toself-learn an efficient storage approach for the storage of streams ofanalog sensor data from an environment.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a probabilisticneural network (PNN), which, in embodiments, may comprise a multi-layer(e.g., four-layer) feed forward neural network, where layers may includeinput layers, hidden layers, pattern/summation layers and an outputlayer. In an embodiment of a PNN algorithm, a parent probabilitydistribution function (PDF) of each class may be approximated, such asby a Parzen window and/or a non-parametric function. Then, using the PDFof each class, the class probability of a new input may be estimated,and Bayes' rule may be employed, such as to allocate it to the classwith the highest posterior probability. A PNN may embody a Bayesiannetwork and may use a statistical algorithm or analytic technique, suchas Kernel Fisher discriminant analysis technique. The PNN may be usedfor classification and pattern recognition in any of a wide range ofembodiments disclosed herein. In one non-limiting example, aprobabilistic neural network may be used to predict a fault condition ofa product or system based on a collection of data inputs from sensorsand instruments for the engine.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a time delayneural network (TDNN), which may comprise a feed forward architecturefor sequential data that recognizes features independent of sequenceposition. In embodiments, to account for time shifts in data, delays areadded to one or more inputs, or between one or more nodes, so thatmultiple data points (from distinct points in time) are analyzedtogether. A time delay neural network may form part of a larger patternrecognition system, such as using a perceptron network. In embodiments,a TDNN may be trained with supervised learning, such as where connectionweights are trained with back propagation or under feedback. Inembodiments, a TDNN may be used to process sensor data from distinctstreams, where time delays are used to align the data streams in time,such as to help understand patterns that involve the understanding ofthe various streams.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a convolutionalneural network (referred to in some cases as a CNN, a ConvNet, a shiftinvariant neural network, or a space invariant neural network), whereinthe units are connected in a pattern similar to the visual cortex of thehuman brain. Neurons may respond to stimuli in a restricted region ofspace, referred to as a receptive field. Receptive fields may partiallyoverlap, such that they collectively cover the entire (e.g., visual)field. Node responses can be calculated mathematically, such as by aconvolution operation, such as using. Multilayer perceptrons that useminimal preprocessing. A convolutional neural network may be used forrecognition within images and video streams, such as for recognizing anindividual, recognizing a marker of a disease condition, or the like.This may include recognizing an individual in a crowd, such as using acamera system disposed on a mobile data collector, such as on a drone ormobile robot. In embodiments, a convolutional neural network may be usedto provide a recommendation based on data inputs, including sensorinputs and other contextual information. In embodiments, a convolutionalneural network may be used for processing inputs, such as for naturallanguage processing of instructions provided by one or more partiesinvolved in a workflow in an environment. In embodiments, aconvolutional neural network may be deployed with a large number ofneurons (e.g., 100,000, 500,000 or more), with multiple (e.g., 4, 5, 6or more) layers, and with many (e.g., millions) of parameters. Aconvolutional neural net may use one or more convolutional nets.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a regulatoryfeedback network, such as for recognizing emergent phenomena (such asnew types of conditions not previously understood in an individual orpopulation of individuals).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a self-organizingmap (SOM), involving unsupervised learning. A set of neurons may learnto map points in an input space to coordinates in an output space. Theinput space can have different dimensions and topology from the outputspace, and the SOM may preserve these while mapping phenomena intogroups.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a learning vectorquantization neural net (LVQ). Prototypical representatives of theclasses may parameterize, together with an appropriate distance measure,in a distance-based classification scheme.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an echo statenetwork (ESN), which may comprise a recurrent neural network with asparsely connected, random hidden layer. The weights of output neuronsmay be changed (e.g., the weights may be trained based on feedback). Inembodiments, an ESN may be used to handle time series patterns, such as,in an example, recognizing a pattern of progression of a process.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a Bi-directional,recurrent neural network (BRNN), such as using a finite sequence ofvalues (e.g., voltage values from a sensor) to predict or label eachelement of the sequence based on both the past and the future context ofthe element. This may be done by adding the outputs of two RNNs, such asone processing the sequence from left to right, the other one from rightto left. The combined outputs are the predictions of target signals,such as ones provided by a teacher or supervisor. A bi-directional RNNmay be combined with a long short-term memory RNN.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a hierarchical RNNthat connects elements in various ways to decompose hierarchicalbehavior, such as into useful subprograms. In embodiments, ahierarchical RNN may be used to manage one or more hierarchicaltemplates for data collection in a social network, a value chainenvironment, or the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a stochasticneural network, which may introduce random variations into the network.Such random variations can be viewed as a form of statistical sampling,such as Monte Carlo sampling or other statistical sampling techniques.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a genetic scalerecurrent neural network. In such embodiments, an RNN (often a LSTM) maybe used where a series may be decomposed into a number of scales whereevery scale informs the primary length between two consecutive points. Afirst order scale consists of a normal RNN, a second order consists ofall points separated by two indices and so on. The Nth order RNNconnects the first and last node. The outputs from all the variousscales may be treated as a committee of members, and the associatedscores may be used genetically for the next iteration.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a committee ofmachines (CoM), comprising a collection of different neural networksthat together “vote” on a given example. Because neural networks maysuffer from local minima, starting with the same architecture andtraining, but using randomly different initial weights often givesdifferent results. A CoM tends to stabilize the result.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an associativeneural network (ASNN), such as involving an extension of a committee ofmachines that combines multiple feed forward neural networks and ak-nearest neighbor technique. It may use the correlation betweenensemble responses as a measure of distance amid the analyzed cases forthe kNN. This corrects the bias of the neural network ensemble. Anassociative neural network may have a memory that can coincide with atraining set. If new data become available, the network instantlyimproves its predictive ability and provides data approximation(self-learns) without retraining. Another important feature of ASNN maybe the possibility to interpret neural network results by analysis ofcorrelations between data cases in the space of models.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an instantaneouslytrained neural network (ITNN), where the weights of the hidden and theoutput layers are mapped directly from training vector data.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a spiking neuralnetwork, which may explicitly consider the timing of inputs. The networkinput and output may be represented as a series of spikes (such as adelta function or more complex shapes). SNNs can process information inthe time domain (e.g., signals that vary over time, such as signalsinvolving dynamic behavior of an individual, a disease condition, ahealth condition, or the like). They may be implemented as recurrentnetworks.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a dynamic neuralnetwork that addresses nonlinear multivariate behavior and includeslearning of time-dependent behavior, such as transient phenomena anddelay effects. Transients may include behavior of progressing states.

In embodiments, cascade correlation may be used as an architecture andsupervised learning algorithm, supplementing adjustment of the weightsin a network of fixed topology. Cascade-correlation may begin with aminimal network, then automatically trains and add new hidden units oneby one, creating a multi-layer structure. Once a new hidden unit hasbeen added to the network, its input-side weights may be frozen. Thisunit then becomes a permanent feature-detector in the network, availablefor producing outputs or for creating other, more complex featuredetectors. The cascade-correlation architecture may learn quickly,determine its own size and topology, and retain the structures it hasbuilt even if the training set changes and requires no back-propagation.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a neuro-fuzzynetwork, such as involving a fuzzy interference system in the body of anartificial neural network. Depending on the type, several layers maysimulate the processes involved in a fuzzy inference, such asfuzzification, inference, aggregation and defuzzification. Embedding afuzzy system in a general structure of a neural net as the benefit ofusing available training methods to find the parameters of a fuzzysystem.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a compositionalpattern-producing network (CPPN), such as a variation of an associativeneural network (ANN) that differs the set of activation functions andhow they are applied. While typical ANNs often contain only sigmoidfunctions (and sometimes Gaussian functions. PPNs can include both typesof functions and many others. Furthermore, CPPNs may be applied acrossthe entire space of possible inputs, so that they can represent acomplete image. Since they are compositions of functions, CPPNs ineffect encode images at infinite resolution and can be sampled for aparticular display at whatever resolution may be optimal. This type ofnetwork can add new patterns without re-training. In embodiments,methods and systems described herein that involve an expert system orself-organization capability may use a one-shot associative memorynetwork, such as by creating a specific memory structure, which assignseach new pattern to an orthogonal plane using adjacently connectedhierarchical arrays.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a hierarchicaltemporal memory (HTM) neural network, such as involving the structuraland algorithmic properties of the neocortex. HTM may use a biomimeticmodel, such as based on memory-prediction. HTM may be used to discoverand infer the high-level causes of observed input patterns andsequences.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a holographicassociative memory (HAM) neural network, which may comprise an analog,correlation-based, associative, stimulus-response system. Informationmay be mapped onto the phase orientation of complex numbers. The memorymay be effective for associative memory tasks, generalization andpattern recognition with changeable attention.

The foregoing neural networks may have a variety of nodes or neurons,which may perform a variety of functions on inputs, such as inputsreceived from sensors or other data sources, including other nodes.Functions may involve weights, features, feature vectors, and the like.Neurons may include perceptrons, neurons that mimic biological functions(such as of the human senses of touch, vision, taste, hearing, andsmell), and the like. Continuous neurons, such as with sigmoidalactivation, may be used in the context of various forms of neural net,such as where back propagation is involved.

In many embodiments, an expert system or neural network may be trained,such as by a human operator or supervisor, or based on a data set,model, or the like. Training may include presenting the neural networkwith one or more training data sets that represent values, such assensor data, event data, parameter data, and other types of data(including the many types described throughout this disclosure), as wellas one or more indicators of an outcome, such as an outcome of aprocess, an outcome of a calculation, an outcome of an event, an outcomeof an activity, or the like. Training may include training inoptimization, such as training a neural network to optimize one or moresystems based on one or more optimization approaches, such as Bayesianapproaches, parametric Bayes classifier approaches, k-nearest-neighborclassifier approaches, iterative approaches, interpolation approaches,Pareto optimization approaches, algorithmic approaches, and the like.Feedback may be provided in a process of variation and selection, suchas with a genetic algorithm that evolves one or more solutions based onfeedback through a series of rounds.

In embodiments, a plurality of neural networks may be deployed in acloud platform that receives data streams and other inputs collected(such as by mobile data collectors) in one or more industrialenvironments and transmitted to the cloud platform over one or morenetworks, including using network coding to provide efficienttransmission. In the cloud platform, optionally using massively parallelcomputational capability, a plurality of different neural networks ofseveral types (including modular forms, structure-adaptive forms,hybrids, and the like) may be used to undertake prediction,classification, control functions, and provide other outputs asdescribed in connection with expert systems disclosed throughout thisdisclosure. The different neural networks may be structured to competewith each other (optionally including the use of evolutionaryalgorithms, genetic algorithms, or the like), such that an appropriatetype of neural network, with appropriate input sets, weights, node typesand functions, and the like, may be selected, such as by an expertsystem, for a specific task involved in a given context, workflow,environment process, system, or the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a feed forwardneural network, which moves information in one direction, such as from adata input, like an analog sensor located on or proximal to anindustrial machine, through a series of neurons or nodes, to an output.Data may move from the input nodes to the output nodes, optionallypassing through one or more hidden nodes, without loops. In embodiments,feedforward neural networks may be constructed with various types ofunits, such as binary McCulloch-Pitts neurons, the simplest of which isa perceptron.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a radial basisfunction (RBF) neural network, which may be preferred in some situationsinvolving interpolation in a multi-dimensional space (such as whereinterpolation is helpful in optimizing a multi-dimensional function,such as for optimizing a data marketplace as described here, optimizingthe efficiency or output of a power generation system, a factory system,or the like, or other situation involving multiple dimensions). Inembodiments, each neuron in the RBF neural network stores an examplefrom a training set as a “prototype.” Linearity involved in thefunctioning of this neural network offers RBF the advantage of nottypically suffering from problems with local minima or maxima.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a radial basisfunction (RBF) neural network, such as one that employs a distancecriterion with respect to a center (e.g., a Gaussian function). A radialbasis function may be applied as a replacement for a hidden layer (suchas a sigmoidal hidden layer transfer) in a multi-layer perceptron. AnRBF network may have two layers, such as the case where an input ismapped onto each RBF in a hidden layer. In embodiments, an output layermay comprise a linear combination of hidden layer values representing,for example, a mean predicted output. The output layer value may providean output that is the same as or similar to that of a regression modelin statistics. In classification problems, the output layer may be asigmoid function of a linear combination of hidden layer values,representing a posterior probability. Performance in both cases is oftenimproved by shrinkage techniques, such as ridge regression in classicalstatistics. This corresponds to a prior belief in small parameter values(and therefore smooth output functions) in a Bayesian framework. RBFnetworks may avoid local minima, because the only parameters that areadjusted in the learning process are the linear mapping from hiddenlayer to output layer. Linearity ensures that the error surface isquadratic and therefore has a single minimum. In regression problems,this can be found in one matrix operation. In classification problems,the fixed non-linearity introduced by the sigmoid output function may behandled using an iteratively re-weighted least squares function or thelike.

RBF networks may use kernel methods such as support vector machines(SVM) and Gaussian processes (where the RBF is the kernel function). Anon-linear kernel function may be used to project the input data into aspace where the learning problem can be solved using a linear model.

In embodiments, an RBF neural network may include an input layer, ahidden layer, and a summation layer. In the input layer, one neuronappears in the input layer for each predictor variable. In the case ofcategorical variables, N−1 neurons are used, where N is the number ofcategories. The input neurons may, in embodiments, standardize the valueranges by subtracting the median and dividing by the interquartilerange. The input neurons may then feed the values to each of the neuronsin the hidden layer. In the hidden layer, a variable number of neuronsmay be used (determined by the training process). Each neuron mayconsist of a radial basis function that is centered on a point with asmany dimensions as a number of predictor variables. The spread (e.g.,radius) of the RBF function may be different for each dimension. Thecenters and spreads may be determined by training. When presented with avector of input values from the input layer, a hidden neuron may computea Euclidean distance of the test case from the neuron's center point andthen apply the RBF kernel function to this distance, such as using thespread values. The resulting value may then be passed to the summationlayer. In the summation layer, the value coming out of a neuron in thehidden layer may be multiplied by a weight associated with the neuronand may add to the weighted values of other neurons. This sum becomesthe output. For classification problems, one output is produced (with aseparate set of weights and summation units) for each target category.The value output for a category is the probability that the case beingevaluated has that category. In training of an RBF, various parametersmay be determined, such as the number of neurons in a hidden layer, thecoordinates of the center of each hidden-layer function, the spread ofeach function in each dimension, and the weights applied to outputs asthey pass to the summation layer. Training may be used by clusteringalgorithms (such as k-means clustering), by evolutionary approaches, andthe like.

In embodiments, a recurrent neural network may have a time-varying,real-valued (more than just zero or one) activation (output). Eachconnection may have a modifiable real-valued weight. Some of the nodesare called labeled nodes, some output nodes, and other hidden nodes. Forsupervised learning in discrete time settings, training sequences ofreal-valued input vectors may become sequences of activations of theinput nodes, one input vector at a time. At each time step, eachnon-input unit may compute its current activation as a nonlinearfunction of the weighted sum of the activations of all units from whichit receives connections. The system can explicitly activate (independentof incoming signals) some output units at certain time steps.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a self-organizingneural network, such as a Kohonen self-organizing neural network, suchas for visualization of views of data, such as low-dimensional views ofhigh-dimensional data. The self-organizing neural network may applycompetitive learning to a set of input data, such as from one or moresensors or other data inputs from or associated with an industrialmachine. In embodiments, the self-organizing neural network may be usedto identify structures in data, such as unlabeled data, such as in datasensed from a range of vibration, acoustic, or other analog sensors inan industrial environment, where sources of the data are unknown (suchas where vibrations may be coming from any of a range of unknownsources). The self-organizing neural network may organize structures orpatterns in the data, such that they can be recognized, analyzed, andlabeled, such as identifying structures as corresponding to vibrationsinduced by the movement of a floor, or acoustic signals created by highfrequency rotation of a shaft of a somewhat distant machine.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a recurrent neuralnetwork, which may allow for a bi-directional flow of data, such aswhere connected units (e.g., neurons or nodes) form a directed cycle.Such a network may be used to model or exhibit dynamic temporalbehavior, such as those involved in dynamic systems including a widevariety of the industrial machines and devices described throughout thisdisclosure, such as a power generation machine operating at variablespeeds or frequencies in variable conditions with variable inputs, arobotic manufacturing system, a refining system, or the like, wheredynamic system behavior involves complex interactions that an operatormay desire to understand, predict, control and/or optimize. For example,the recurrent neural network may be used to anticipate the state (suchas a maintenance state, a fault state, an operational state, or thelike), of an industrial machine, such as one performing a dynamicprocess or action. In embodiments, the recurrent neural network may useinternal memory to process a sequence of inputs, such as from othernodes and/or from sensors and other data inputs from the industrialenvironment, of the various types described herein. In embodiments, therecurrent neural network may also be used for pattern recognition, suchas for recognizing an industrial machine based on a sound signature, aheat signature, a set of feature vectors in an image, a chemicalsignature, or the like. In a non-limiting example, a recurrent neuralnetwork may recognize a shift in an operational mode of a turbine, agenerator, a motor, a compressor, or the like (such as a gear shift) bylearning to classify the shift from a training data set consisting of astream of data from tri-axial vibration sensors and/or acoustic sensorsapplied to one or more of such machines.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a modular neuralnetwork, which may comprise a series of independent neural networks(such as ones of various types described herein) that are moderated byan intermediary. Each of the independent neural networks in the modularneural network may work with separate inputs, accomplishing subtasksthat make up the task the modular network as a whole is intended toperform. For example, a modular neural network may comprise a recurrentneural network for pattern recognition, such as to recognize what typeof industrial machine is being sensed by one or more sensors that areprovided as input channels to the modular network and an RBF neuralnetwork for optimizing the behavior of the machine once understood. Theintermediary may accept inputs of each of the individual neuralnetworks, process them, and create output for the modular neuralnetwork, such an appropriate control parameter, a prediction of state,or the like.

Combinations among any of the pairs, triplets, or larger combinations,of the various neural network types described herein, are encompassed bythe present disclosure. This may include combinations where an expertsystem uses one neural network for recognizing a pattern (e.g., apattern indicating a problem or fault condition) and a different neuralnetwork for self-organizing an activity or work flow based on therecognized pattern (such as providing an output governing autonomouscontrol of a system in response to the recognized condition or pattern).This may also include combinations where an expert system uses oneneural network for classifying an item (e.g., identifying a machine, acomponent, or an operational mode) and a different neural network forpredicting a state of the item (e.g., a fault state, an operationalstate, an anticipated state, a maintenance state, or the like). Modularneural networks may also include situations where an expert system usesone neural network for determining a state or context (such as a stateof a machine, a process, a work flow, a marketplace, a storage system, anetwork, a data collector, or the like) and a different neural networkfor self-organizing a process involving the state or context (e.g., adata storage process, a network coding process, a network selectionprocess, a data marketplace process, a power generation process, amanufacturing process, a refining process, a digging process, a boringprocess, or other process described herein).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a physical neuralnetwork where one or more hardware elements are used to perform orsimulate neural behavior. In embodiments, one or more hardware neuronsmay be configured to stream voltage values that represent analogvibration sensor data voltage values, to calculate velocity informationfrom analog sensor inputs representing acoustic, vibration or otherdata, to calculation acceleration information from sensor inputsrepresenting acoustic, vibration, or other data, or the like. One ormore hardware nodes may be configured to stream output data resultingfrom the activity of the neural net. Hardware nodes, which may compriseone or more chips, microprocessors, integrated circuits, programmablelogic controllers, application-specific integrated circuits,field-programmable gate arrays, or the like, may be provided to optimizethe speed, input/output efficiency, energy efficiency, signal to noiseratio, or other parameter of some part of a neural net of any of thetypes described herein. Hardware nodes may include hardware foracceleration of calculations (such as dedicated processors forperforming basic or more sophisticated calculations on input data toprovide outputs, dedicated processors for filtering or compressing data,dedicated processors for decompressing data, dedicated processors forcompression of specific file or data types (e.g., for handling imagedata, video streams, acoustic signals, vibration data, thermal images,heat maps, or the like), and the like. A physical neural network may beembodied in a data collector, such as a mobile data collector describedherein, including one that may be reconfigured by switching or routinginputs in varying configurations, such as to provide different neuralnet configurations within the data collector for handling differenttypes of inputs (with the switching and configuration optionally undercontrol of an expert system, which may include a software-based neuralnet located on the data collector or remotely). A physical, or at leastpartially physical, neural network may include physical hardware nodeslocated in a storage system, such as for storing data within anindustrial machine or in an industrial environment, such as foraccelerating input/output functions to one or more storage elements thatsupply data to or take data from the neural net. A physical, or at leastpartially physical, neural network may include physical hardware nodeslocated in a network, such as for transmitting data within, to or froman industrial environment, such as for accelerating input/outputfunctions to one or more network nodes in the net, accelerating relayfunctions, or the like. In embodiments, of a physical neural network, anelectrically adjustable resistance material may be used for emulatingthe function of a neural synapse. In embodiments, the physical hardwareemulates the neurons, and software emulates the neural network betweenthe neurons. In embodiments, neural networks complement conventionalalgorithmic computers. They are versatile and can be trained to performappropriate functions without the need for any instructions, such asclassification functions, optimization functions, pattern recognitionfunctions, control functions, selection functions, evolution functions,and others.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a multilayeredfeed forward neural network, such as for complex pattern classificationof one or more items, phenomena, modes, states, or the like. Inembodiments, a multilayered feedforward neural network may be trained byan optimization technique, such as a genetic algorithm, such as toexplore a large and complex space of options to find an optimum, ornear-optimum, global solution. For example, one or more geneticalgorithms may be used to train a multilayered feedforward neuralnetwork to classify complex phenomena, such as to recognize complexoperational modes of industrial machines, such as modes involvingcomplex interactions among machines (including interference effects,resonance effects, and the like), modes involving non-linear phenomena,such as impacts of variable speed shafts, which may make analysis ofvibration and other signals difficult, modes involving critical faults,such as where multiple, simultaneous faults occur, making root causeanalysis difficult, and others. In embodiments, a multilayered feedforward neural network may be used to classify results from ultrasonicmonitoring or acoustic monitoring of an industrial machine, such asmonitoring an interior set of components within a housing, such as motorcomponents, pumps, valves, fluid handling components, and many others,such as in refrigeration systems, refining systems, reactor systems,catalytic systems, and others.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a feedforward,back-propagation multi-layer perceptron (MLP) neural network, such asfor handling one or more remote sensing applications, such as for takinginputs from sensors distributed throughout various industrialenvironments. In embodiments, the MLP neural network may be used forclassification of physical environments, such as mining environments,exploration environments, drilling environments, and the like, includingclassification of geological structures (including underground featuresand above ground features), classification of materials (includingfluids, minerals, metals, and the like), and other problems. This mayinclude fuzzy classification.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use astructure-adaptive neural network, where the structure of a neuralnetwork is adapted, such as based on a rule, a sensed condition, acontextual parameter, or the like. For example, if a neural network doesnot converge on a solution, such as classifying an item or arriving at aprediction, when acting on a set of inputs after some amount oftraining, the neural network may be modified, such as from a feedforwardneural network to a recurrent neural network, such as by switching datapaths between some subset of nodes from unidirectional to bi-directionaldata paths. The structure adaptation may occur under control of anexpert system, such as to trigger adaptation upon occurrence of atrigger, rule or event, such as recognizing occurrence of a threshold(such as an absence of a convergence to a solution within a given amountof time) or recognizing a phenomenon as requiring different oradditional structure (such as recognizing that a system is varyingdynamically or in a non-linear fashion). In one non-limiting example, anexpert system may switch from a simple neural network structure like afeedforward neural network to a more complex neural network structurelike a recurrent neural network, a convolutional neural network, or thelike upon receiving an indication that a continuously variabletransmission is being used to drive a generator, turbine, or the like ina system being analyzed.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an autoencoder,autoassociator or Diabolo neural network, which may be similar to amultilayer perceptron (“MLP”) neural network, such as where there may bean input layer, an output layer and one or more hidden layers connectingthem. However, the output layer in the auto-encoder may have the samenumber of units as the input layer, where the purpose of the MLP neuralnetwork is to reconstruct its own inputs (rather than just emitting atarget value). Therefore, the auto encoders may operate as anunsupervised learning model. An auto encoder may be used, for example,for unsupervised learning of efficient codings, such as fordimensionality reduction, for learning generative models of data, andthe like. In embodiments, an auto-encoding neural network may be used toself-learn an efficient network coding for transmission of analog sensordata from an industrial machine over one or more networks. Inembodiments, an auto-encoding neural network may be used to self-learnan efficient storage approach for storage of streams of analog sensordata from an industrial environment.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a probabilisticneural network (“PNN”), which, in embodiments, may comprise amulti-layer (e.g., four-layer) feedforward neural network, where layersmay include input layers, hidden layers, pattern/summation layers and anoutput layer. In an embodiment of a PNN algorithm, a parent probabilitydistribution function (PDF) of each class may be approximated, such asby a Parzen window and/or a non-parametric function. Then, using the PDFof each class, the class probability of a new input is estimated, andBayes' rule may be employed, such as to allocate it to the class withthe highest posterior probability. A PNN may embody a Bayesian networkand may use a statistical algorithm or analytic technique, such asKernel Fisher discriminant analysis technique. The PNN may be used forclassification and pattern recognition in any of a wide range ofembodiments disclosed herein. In one non-limiting example, aprobabilistic neural network may be used to predict a fault condition ofan engine based on a collection of data inputs from sensors andinstruments for the engine.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a time delayneural network (TDNN), which may comprise a feedforward architecture forsequential data that recognizes features independent of sequenceposition. In embodiments, to account for time shifts in data, delays areadded to one or more inputs, or between one or more nodes, so thatmultiple data points (from distinct points in time) are analyzedtogether. A time delay neural network may form part of a larger patternrecognition system, such as using a perceptron network. In embodiments,a TDNN may be trained with supervised learning, such as where connectionweights are trained with back propagation or under feedback. Inembodiments, a TDNN may be used to process sensor data from distinctstreams, such as a stream of velocity data, a stream of accelerationdata, a stream of temperature data, a stream of pressure data, and thelike, where time delays are used to align the data streams in time, suchas to help understand patterns that involve understanding of the variousstreams (e.g., where increases in pressure and acceleration occur as anindustrial machine overheats).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a convolutionalneural network (referred to in some cases as a CNN, a ConvNet, a shiftinvariant neural network, or a space invariant neural network), whereinthe units are connected in a pattern similar to the visual cortex of thehuman brain. Neurons may respond to stimuli in a restricted region ofspace, referred to as a receptive field. Receptive fields may partiallyoverlap, such that they collectively cover the entire (e.g., visual)field. Node responses can be calculated mathematically, such as by aconvolution operation, such as using multilayer perceptrons that useminimal preprocessing. A convolutional neural network may be used forrecognition within images and video streams, such as for recognizing atype of machine in a large environment using a camera system disposed ona mobile data collector, such as on a drone or mobile robot. Inembodiments, a convolutional neural network may be used to provide arecommendation based on data inputs, including sensor inputs and othercontextual information, such as recommending a route for a mobile datacollector. In embodiments, a convolutional neural network may be usedfor processing inputs, such as for natural language processing ofinstructions provided by one or more parties involved in a workflow inan environment. In embodiments, a convolutional neural network may bedeployed with a large number of neurons (e.g., 100,000, 500,000 ormore), with multiple (e.g., 4, 5, 6 or more) layers, and with many(e.g., millions) parameters. A convolutional neural net may use one ormore convolutional nets.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a regulatoryfeedback network, such as for recognizing emergent phenomena (such asnew types of faults not previously understood in an industrialenvironment).

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a self-organizingmap (“SOM”), involving unsupervised learning. A set of neurons may learnto map points in an input space to coordinates in an output space. Theinput space can have different dimensions and topology from the outputspace, and the SOM may preserve these while mapping phenomena intogroups.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a learning vectorquantization neural net (“LVQ”). Prototypical representatives of theclasses may parameterize, together with an appropriate distance measure,in a distance-based classification scheme.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an echo statenetwork (“ESN”), which may comprise a recurrent neural network with asparsely connected, random hidden layer. The weights of output neuronsmay be changed (e.g., the weights may be trained based on feedback). Inembodiments, an ESN may be used to handle time series patterns, such as,in an example, recognizing a pattern of events associated with a gearshift in an industrial turbine, generator, or the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a bi-directional,recurrent neural network (“BRNN”), such as using a finite sequence ofvalues (e.g., voltage values from a sensor) to predict or label eachelement of the sequence based on both the past and the future context ofthe element. This may be done by adding the outputs of two RNNs, such asone processing the sequence from left to right, the other one from rightto left. The combined outputs are the predictions of target signals,such as those provided by a teacher or supervisor. A bi-directional RNNmay be combined with a long short-term memory RNN.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a hierarchical RNNthat connects elements in various ways to decompose hierarchicalbehavior, such as into useful subprograms. In embodiments, ahierarchical RNN may be used to manage one or more hierarchicaltemplates for data collection in an industrial environment.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a stochasticneural network, which may introduce random variations into the network.Such random variations can be viewed as a form of statistical sampling,such as Monte Carlo sampling.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a genetic scalerecurrent neural network. In such embodiments, a RNN (often a LSTM) isused where a series is decomposed into a number of scales where everyscale informs the primary length between two consecutive points. A firstorder scale consists of a normal RNN, a second order consists of allpoints separated by two indices and so on. The Nth order RNN connectsthe first and last node. The outputs from all the various scales may betreated as a committee of members, and the associated scores may be usedgenetically for the next iteration.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a committee ofmachines (“CoM”), comprising a collection of different neural networksthat together “vote” on a given example. Because neural networks maysuffer from local minima, starting with the same architecture andtraining, but using randomly different initial weights often givesdifferent results. A CoM tends to stabilize the result.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an associativeneural network (“ASNN”), such as involving an extension of committee ofmachines that combines multiple feed forward neural networks and ak-nearest neighbor technique. It may use the correlation betweenensemble responses as a measure of distance amid the analyzed cases forthe kNN. This corrects the bias of the neural network ensemble. Anassociative neural network may have a memory that can coincide with atraining set. If new data become available, the network instantlyimproves its predictive ability and provides data approximation(self-learns) without retraining. Another important feature of ASNN isthe possibility to interpret neural network results by analysis ofcorrelations between data cases in the space of models.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use an instantaneouslytrained neural network (“ITNN”), where the weights of the hidden and theoutput layers are mapped directly from training vector data.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a spiking neuralnetwork, which may explicitly consider the timing of inputs. The networkinput and output may be represented as a series of spikes (such as adelta function or more complex shapes). SNNs can process information inthe time domain (e.g., signals that vary over time, such as signalsinvolving dynamic behavior of industrial machines). They are oftenimplemented as recurrent networks.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a dynamic neuralnetwork that addresses nonlinear multivariate behavior and includeslearning of time-dependent behavior, such as transient phenomena anddelay effects. Transients may include behavior of shifting industrialcomponents, such as variable speeds of rotating shafts or other rotatingcomponents.

In embodiments, cascade correlation may be used as an architecture andsupervised learning algorithm, supplementing adjustment of the weightsin a network of fixed topology. Cascade-correlation may begin with aminimal network, then automatically trains and adds new hidden units oneby one, creating a multi-layer structure. Once a new hidden unit hasbeen added to the network, its input-side weights may be frozen. Thisunit then becomes a permanent feature-detector in the network, availablefor producing outputs or for creating other, more complex featuredetectors. The cascade-correlation architecture may learn quickly,determine its own size and topology, and retain the structures it hasbuilt even if the training set changes and requires no back-propagation.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a neuro-fuzzynetwork, such as involving a fuzzy inference system in the body of anartificial neural network. Depending on the type, several layers maysimulate the processes involved in a fuzzy inference, such asfuzzification, inference, aggregation and defuzzification. Embedding afuzzy system in a general structure of a neural net as the benefit ofusing available training methods to find the parameters of a fuzzysystem.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a compositionalpattern-producing network (“CPPN”), such as a variation of anassociative neural network (“ANN”) that differs the set of activationfunctions and how they are applied. While typical ANNs often containonly sigmoid functions (and sometimes Gaussian functions), CPPNs caninclude both types of functions and many others. Furthermore, CPPNs maybe applied across the entire space of possible inputs, so that they canrepresent a complete image. Since they are compositions of functions,CPPNs in effect encode images at infinite resolution and can be sampledfor a particular display at whatever resolution is optimal.

This type of network can add new patterns without re-training. Inembodiments, methods and systems described herein that involve an expertsystem or self-organization capability may use a one-shot associativememory network, such as by creating a specific memory structure, whichassigns each new pattern to an orthogonal plane using adjacentlyconnected hierarchical arrays.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a hierarchicaltemporal memory (“HTM”) neural network, such as involving the structuraland algorithmic properties of the neocortex. HTM may use a biomimeticmodel based on memory-prediction theory. HTM may be used to discover andinfer the high-level causes of observed input patterns and sequences.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a holographicassociative memory (“HAM”) neural network, which may comprise an analog,correlation-based, associative, stimulus-response system. Informationmay be mapped onto the phase orientation of complex numbers. The memoryis effective for associative memory tasks, generalization and patternrecognition with changeable attention.

Intelligence System

FIG. 104 illustrates an example intelligence services system 8800 (alsoreferred to as “intelligence services”) according to some embodiments ofthe present disclosure. In embodiments, the intelligence services 8800provides a framework for providing intelligence services to one or moreintelligence service clients 8836. In some embodiments, the intelligenceservices 8800 framework may be adapted to be at least partiallyreplicated in respective intelligence clients 8836 (e.g., VCN controltowers and/or various VCN entities). In these embodiments, an individualclient 8836 may include some or all of the capabilities of theintelligence services 8800, whereby the intelligence services 8800 isadapted for the specific functions performed by the subsystems of theintelligence client. Additionally or alternatively, in some embodiments,the intelligence services 8800 may be implemented as a set ofmicroservices, such that different intelligence clients 8836 mayleverage the intelligence services 8800 via one or more APIs exposed tothe intelligence clients. In these embodiments, the intelligenceservices 8800 may be configured to perform various types of intelligenceservices that may be adapted for different intelligence clients 8836. Ineither of these configurations, an intelligence service client 8836 mayprovide an intelligence request to the intelligence services 8800,whereby the request is to perform a specific intelligence task (e.g., adecision, a recommendation, a report, an instruction, a classification,a prediction, a training action, an NLP request, or the like). Inresponse, the intelligence services 8800 executes the requestedintelligence task and returns a response to the intelligence serviceclient 8836. Additionally or alternatively, in some embodiments, theintelligence services 8800 may be implemented using one or morespecialized chips that are configured to provide AI assistedmicroservices such as image processing, diagnostics, location andorientation, chemical analysis, data processing, and so forth. Examplesof AI-enabled chips are discussed elsewhere in the disclosure.

In embodiments, an intelligence services 8800 may include anintelligence service controller 8802 and artificial intelligence (AI)modules 8804. In embodiments, an artificial intelligence services 8800receives an intelligence request from an intelligence service client8836 and any required data to process the request from the intelligenceservice client 8836. In response to the request and the specific data,one or more implicated artificial intelligence modules 8804 perform theintelligence task and output an “intelligence response”. Examples ofintelligence modules 8804 responses may include a decision (e.g., acontrol instruction, a proposed action, machine-generated text, and/orthe like), a prediction (e.g., a predicted meaning of a text snippet, apredicted outcome associated with a proposed action, a predicted faultcondition, and/or the like), a classification (e.g., a classification ofan object in an image, a classification of a spoken utterance, aclassified fault condition based on sensor data, and/or the like),and/or other suitable outputs of an artificial intelligence system.

In embodiments, artificial intelligence modules 8804 may include an MLmodule 8812, a rules-based module 8828, an analytics module 8818, an RPAmodule 8816, a digital twin module 8820, a machine vision module 8822,an NLP module 8824, and/or a neural network module 8814. It isappreciated that the foregoing are non-limiting examples of artificialintelligence modules, and that some of the modules may be included orleveraged by other artificial intelligence modules. For example, the NLPmodule 8824 and the machine vision module 8822 may leverage differentneural networks that are part of the neural network module 8814 inperformance of their respective functions.

It is further noted that in some scenarios, artificial intelligencemodules 8804 themselves may also be intelligence clients 8836. Forexample, a rules-based intelligence module 8828 may request anintelligence task from an ML module 8812 or a neural networkF41 module8814, such as requesting a classification of an object appearing in avideo and/or a motion of the object. In this example, the rules-basedintelligence module 8828 may be an intelligence service client 8836 thatuses the classification to determine whether to take a specified action.In another example, a machine vision module 8822 may request a digitaltwin of a specified environment from a digital twin module 8820, suchthat the ML module 8812 may request specific data from the digital twinas features to train a machine-learned model that is trained for aspecific environment.

In embodiments, an intelligence task may require specific types of datato respond to the request. For example, a machine vision task requiresone or more images (and potentially other data) to classify objectsappearing in an image or set of images, to determine features within theset of images (such as locations of items, presence of faces, symbols orinstructions, expressions, parameters of motion, changes in status, andmany others), and the like. In another example, an NLP task requiresaudio of speech and/or text data (and potentially other data) todetermine a meaning or other element of the speech and/or text. In yetanother example, an AI-based control task (e.g., a decision on movementof a robot) may require environment data (e.g., maps, coordinates ofknown obstacles, images, and/or the like) and/or a motion plan to make adecision as to how to control the motion of a robot. In a platform-levelexample, an analytics-based reporting task may require data from anumber of different databases to generate a report. Thus, inembodiments, tasks that can be performed by an intelligence services8800 may require, or benefit from, specific intelligence service inputs8832. In some embodiments, an intelligence services 8800 may beconfigured to receive and/or request specific data from the intelligenceservice inputs 8832 to perform a respective intelligence task.Additionally or alternatively, the requesting intelligence serviceclient 8836 may provide the specific data in the request. For instance,the intelligence services 8800 may expose one or more APIs to theintelligence clients 8836, whereby a requesting client 8836 provides thespecific data in the request via the API. Examples of intelligenceservice inputs may include, but are not limited to, sensors that providesensor data, video streams, audio streams, databases, data feeds, humaninput, and/or other suitable data.

In embodiments, intelligence modules 8804 includes and provides accessto an ML module 8812 that may be integrated into or be accessed by oneor more intelligence clients 8836. In embodiments, the ML module 8812may provide machine-based learning capabilities, features, functions,and algorithms for use by an intelligence service client 8836 such astraining ML models, leveraging ML models, reinforcing ML models,performing various clustering techniques, feature extraction, and/or thelike. In an example, a machine learning module 8812 may provide machinelearning computing, data storage, and feedback infrastructure to asimulation system (e.g., as described above). The machine learningmodule 8812 may also operate cooperatively with other modules, such asthe rules-based module 8828, the machine vision module 8822, the RPAmodule 8816, and/or the like.

The machine learning module 8812 may define one or more machine learningmodels for performing analytics, simulation, decision making, andpredictive analytics related to data processing, data analysis,simulation creation, and simulation analysis of one or more componentsor subsystems of an intelligence service client 8836. In embodiments,the machine learning models are algorithms and/or statistical modelsthat perform specific tasks without using explicit instructions, relyinginstead on patterns and inference. The machine learning models build oneor more mathematical models based on training data to make predictionsand/or decisions without being explicitly programmed to perform thespecific tasks. In example implementations, machine learning models mayperform classification, prediction, regression, clustering, anomalydetection, recommendation generation, and/or other tasks.

In embodiments, the machine learning models may perform various types ofclassification based on the input data. Classification is a predictivemodeling problem where a class label is predicted for a given example ofinput data. For example, machine learning models can perform binaryclassification, multi-class or multi-label classification. Inembodiments, the machine-learning model may output “confidence scores”that are indicative of a respective confidence associated withclassification of the input into the respective class. In embodiments,the confidence scores can be compared to one or more thresholds torender a discrete categorical prediction. In embodiments, only a certainnumber of classes (e.g., one) with the relatively largest confidencescores can be selected to render a discrete categorical prediction.

In embodiments, machine learning models may output a probabilisticclassification. For example, machine learning models may predict, givena sample input, a probability distribution over a set of classes. Thus,rather than outputting only the most likely class to which the sampleinput should belong, machine learning models can output, for each class,a probability that the sample input belongs to such class. Inembodiments, the probability distribution over all possible classes cansum to one. In embodiments, a Softmax function, or other type offunction or layer can be used to turn a set of real values respectivelyassociated with the possible classes to a set of real values in therange (0, 1) that sum to one. In embodiments, the probabilities providedby the probability distribution can be compared to one or morethresholds to render a discrete categorical prediction. In embodiments,only a certain number of classes (e.g., one) with the relatively largestpredicted probability can be selected to render a discrete categoricalprediction.

In embodiments, machine learning models can perform regression toprovide output data in the form of a continuous numeric value. Asexamples, machine learning models can perform linear regression,polynomial regression, or nonlinear regression. As described, inembodiments, a Softmax function or other function or layer can be usedto squash a set of real values respectively associated with a two ormore possible classes to a set of real values in the range (0, 1) thatsum to one. For example, machine learning models can perform linearregression, polynomial regression, or nonlinear regression. As examples,machine learning models can perform simple regression or multipleregression. As described above, in some implementations, a Softmaxfunction or other function or layer can be used to squash a set of realvalues respectively associated with a two or more possible classes to aset of real values in the range (0, 1) that sum to one.

In embodiments, machine learning models may perform various types ofclustering. For example, machine learning models may identify one ormore previously-defined clusters to which the input data most likelycorresponds. In some implementations in which machine learning modelsperforms clustering, machine learning models can be trained usingunsupervised learning techniques.

In embodiments, machine learning models may perform anomaly detection oroutlier detection. For example, machine learning models can identifyinput data that does not conform to an expected pattern or othercharacteristic (e.g., as previously observed from previous input data).As examples, the anomaly detection can be used for fraud detection orsystem failure detection.

In some implementations, machine learning models can provide output datain the form of one or more recommendations. For example, machinelearning models can be included in a recommendation system or engine. Asan example, given input data that describes previous outcomes forcertain entities (e.g., a score, ranking, or rating indicative of anamount of success or enjoyment), machine learning models can output asuggestion or recommendation of one or more additional entities that,based on the previous outcomes, are expected to have a desired outcome

As described above, machine learning models can be or include one ormore of various different types of machine-learned models. Examples ofsuch different types of machine-learned models are provided below forillustration. One or more of the example models described below can beused (e.g., combined) to provide the output data in response to theinput data. Additional models beyond the example models provided belowcan be used as well.

In some implementations, machine learning models can be or include oneor more classifier models such as, for example, linear classificationmodels; quadratic classification models; etc. Machine learning modelsmay be or include one or more regression models such as, for example,simple linear regression models; multiple linear regression models;logistic regression models; stepwise regression models; multivariateadaptive regression splines; locally estimated scatterplot smoothingmodels; etc.

In some examples, machine learning models can be or include one or moredecision tree-based models such as, for example, classification and/orregression trees; chi-squared automatic interaction detection decisiontrees; decision stumps; conditional decision trees; etc.

Machine learning models may be or include one or more kernel machines.In some implementations, machine learning models can be or include oneor more support vector machines. Machine learning models may be orinclude one or more instance-based learning models such as, for example,learning vector quantization models; self-organizing map models; locallyweighted learning models; etc. In some implementations, machine learningmodels can be or include one or more nearest neighbor models such as,for example, k-nearest neighbor classifications models; k-nearestneighbors regression models; etc. Machine learning models can be orinclude one or more Bayesian models such as, for example, naïve Bayesmodels; Gaussian naïve Bayes models; multinomial naïve Bayes models;averaged one-dependence estimators; Bayesian networks; Bayesian beliefnetworks; hidden Markov models; etc.

Machine learning models may include one or more clustering models suchas, for example, k-means clustering models; k-medians clustering models;expectation maximization models; hierarchical clustering models; etc.

In some implementations, machine learning models can perform one or moredimensionality reduction techniques such as, for example, principalcomponent analysis; kernel principal component analysis; graph-basedkernel principal component analysis; principal component regression;partial least squares regression; Sammon mapping; multidimensionalscaling; projection pursuit; linear discriminant analysis; mixturediscriminant analysis; quadratic discriminant analysis; generalizeddiscriminant analysis; flexible discriminant analysis; autoencoding;etc.

In some implementations, machine learning models can perform or besubjected to one or more reinforcement learning techniques such asMarkov decision processes; dynamic programming; Q functions orQ-learning; value function approaches; deep Q-networks; differentiableneural computers; asynchronous advantage actor-critics; deterministicpolicy gradient; etc.

In embodiments, artificial intelligence modules 8804 may include and/orprovide access to a neural network module 8814. In embodiments, theneural network module 8814 is configured to train, deploy, and/orleverage artificial neural networks (or “neural networks”) on behalf ofan intelligence service client 8836. It is noted that in thedescription, the term machine learning model may include neuralnetworks, and as such, the neural network module 8814 may be part of themachine learning module 8812. In embodiments, the neural network module8814 may be configured to train neural networks that may be used by theintelligence clients 8836. Non-limiting examples of different types ofneural networks may include any of the neural network types describedthroughout this disclosure and the documents incorporated herein byreference, including without limitation convolutional neural networks(CNN), deep convolutional neural networks (DCN), feed forward neuralnetworks (including deep feed forward neural networks), recurrent neuralnetworks (RNN) (including without limitation gated RNNs), long/shortterm memory (LTSM) neural networks, and the like, as well as hybrids orcombinations of the above, such as deployed in series, in parallel, inacyclic (e.g., directed graph-based) flows, and/or in more complex flowsthat may include intermediate decision nodes, recursive loops, and thelike, where a given type of neural network takes inputs from a datasource or other neural network and provides outputs that are includedwithin the input sets of another neural network until a flow iscompleted and a final output is provided. In embodiments, the neuralnetwork module 8814 may be leveraged by other artificial intelligencemodules 8804, such as the machine vision module 8822, the NLP module8824, the rules-based module 8828, the digital twin module 8826, and soon. Example applications of the neural network module 8814 are describedthroughout the disclosure.

A neural network includes a group of connected nodes, which also can bereferred to as neurons or perceptrons. A neural network can be organizedinto one or more layers. Neural networks that include multiple layerscan be referred to as “deep” networks. A deep network can include aninput layer, an output layer, and one or more hidden layers positionedbetween the input layer and the output layer. The nodes of the neuralnetwork can be connected or non-fully connected.

In embodiments, the neural networks can be or include one or more feedforward neural networks. In feed forward networks, the connectionsbetween nodes do not form a cycle. For example, each connection canconnect a node from an earlier layer to a node from a later layer.

In embodiments, the neural networks can be or include one or morerecurrent neural networks. In some instances, at least some of the nodesof a recurrent neural network can form a cycle. Recurrent neuralnetworks can be especially useful for processing input data that issequential in nature. In particular, in some instances, a recurrentneural network can pass or retain information from a previous portion ofthe input data sequence to a subsequent portion of the input datasequence through the use of recurrent or directed cyclical nodeconnections.

In some examples, sequential input data can include time-series data(e.g., sensor data versus time or imagery captured at different times).For example, a recurrent neural network can analyze sensor data versustime to detect or predict a swipe direction, to perform handwritingrecognition, etc. Sequential input data may include words in a sentence(e.g., for natural language processing, speech detection or processing,etc.); notes in a musical composition; sequential actions taken by auser (e.g., to detect or predict sequential application usage);sequential object states; etc. In some example embodiments, recurrentneural networks include long short-term (LSTM) recurrent neuralnetworks; gated recurrent units; bi-direction recurrent neural networks;continuous time recurrent neural networks; neural history compressors;echo state networks; Elman networks; Jordan networks; recursive neuralnetworks; Hopfield networks; fully recurrent networks;sequence-to-sequence configurations; etc.

In some examples, neural networks can be or include one or morenon-recurrent sequence-to-sequence models based on self-attention, suchas Transformer networks. Details of an exemplary transformer network canbe found athttp://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf.

In embodiments, the neural networks can be or include one or moreconvolutional neural networks. In some instances, a convolutional neuralnetwork can include one or more convolutional layers that performconvolutions over input data using learned filters. Filters can also bereferred to as kernels. Convolutional neural networks can be especiallyuseful for vision problems such as when the input data includes imagerysuch as still images or video. However, convolutional neural networkscan also be applied for natural language processing.

In embodiments, the neural networks can be or include one or moregenerative networks such as, for example, generative adversarialnetworks. Generative networks can be used to generate new data such asnew images or other content.

In embodiments, the neural networks may be or include autoencoders. Insome instances, the aim of an autoencoder is to learn a representation(e.g., a lower-dimensional encoding) for a set of data, typically forthe purpose of dimensionality reduction. For example, in some instances,an autoencoder can seek to encode the input data and then provide outputdata that reconstructs the input data from the encoding. Recently, theautoencoder concept has become more widely used for learning generativemodels of data. In some instances, the autoencoder can includeadditional losses beyond reconstructing the input data.

In embodiments, the neural networks may be or include one or more otherforms of artificial neural networks such as, for example, deep Boltzmannmachines; deep belief networks; stacked autoencoders; etc. Any of theneural networks described herein can be combined (e.g., stacked) to formmore complex networks.

FIG. 105 illustrates an example neural network with multiple layers.Neural network 8840 may include an input layer, a hidden layer, and anoutput layer with each layer comprising a plurality of nodes or neuronsthat respond to different combinations of inputs from the previouslayers. The connections between the neurons have numeric weights thatdetermine how much relative effect an input has on the output value ofthe node in question. Input layer may include a plurality of input nodes8842, 8844, 8846, 8848 and 8850 that may provide information from theoutside world or input data (e.g., sensor data, image data, text data,audio data, etc.) to the neural network 8840. The input data may be fromdifferent sources and may include library data x1, simulation data x2,user input data x3, training data x4 and outcome data x5. The inputnodes 8842, 8844, 8846, 8848 and 8850 may pass on the information to thenext layer, and no computation may be performed by the input nodes.Hidden layers may include a plurality of nodes, such as nodes 8852,8854, and 8856. The nodes in the hidden layer 8852, 8854, and 8856 mayprocess the information from the input layer based on the weights of theconnections between the input layer and the hidden layer and transferinformation to the output layer. Output layer may include an output node8858 which processes information based on the weights of the connectionsbetween the hidden layer and the output layer and is responsible forcomputing and transferring information from the network to the outsideworld, such as recognizing certain objects or activities, or predictinga condition or an action.

In embodiments, a neural network 8840 may include two or more hiddenlayers and may be referred to as a deep neural network. The layers areconstructed so that the first layer detects a set of primitive patternsin the input (e.g., image) data, the second layer detects patterns ofpatterns and the third layer detects patterns of those patterns. In someembodiments, a node in the neural network 8840 may have connections toall nodes in the immediately preceding layer and the immediate nextlayer. Thus, the layers may be referred to as fully-connected layers. Insome embodiments, a node in the neural network 8840 may have connectionsto only some of the nodes in the immediately preceding layer and theimmediate next layer. Thus, the layers may be referred to assparsely-connected layers. Each neuron in the neural network consists ofa weighted linear combination of its inputs and the computation on eachneural network layer may be described as a multiplication of an inputmatrix and a weight matrix. A bias matrix is then added to the resultingproduct matrix to account for the threshold of each neuron in the nextlevel. Further, an activation function is applied to each resultantvalue, and the resulting values are placed in the matrix for the nextlayer. Thus, the output from a node i in the neural network may berepresented as:

yi=f(Σxiwi+bi)

where f is the activation function, Σxiwi is the weighted sum of inputmatrix and bi is the bias matrix.

The activation function determines the activity level or excitationlevel generated in the node as a result of an input signal of aparticular size. The purpose of the activation function is to introducenon-linearity into the output of a neural network node because mostreal-world functions are non-linear and it is desirable that the neuronscan learn these non-linear representations. Several activation functionsmay be used in an artificial neural network. One example activationfunction is the sigmoid function σ(x), which is a continuous S-shapedmonotonically increasing function that asymptotically approaches fixedvalues as the input approaches plus or minus infinity. The sigmoidfunction σ(x) takes a real-valued input and transforms it into a valuebetween 0 and 1:

σ(x)=1/(1+exp(−x)).

Another example activation function is the tanh function, which takes areal-valued input and transforms it into a value within the range of[−1, 1]:

tanh(x)=2σ(2x)−1

A third example activation function is the rectified linear unit (ReLU)function. The ReLU function takes a real-valued input and thresholds itabove zero (i.e., replacing negative values with zero):

ƒ(x)=max(0,x).

It will be apparent that the above activation functions are provided asexamples and in various embodiments, neural network 8840 may utilize avariety of activation functions including (but not limited to) identity,binary step, logistic, soft step, tan h, arctan, softsign, rectifiedlinear unit (ReLU), leaky rectified linear unit, parameteric rectifiedlinear unit, randomized leaky rectified linear unit, exponential linearunit, s-shaped rectified linear activation unit, adaptive piecewiselinear, softplus, bent identity, softexponential, sinusoid, sinc,gaussian, softmax, maxout, and/or a combination of activation functions.

In the example shown in FIG. 105 , nodes 8842, 8844, 8846, 8848 and 8850in the input layer may take external inputs x1, x2, x3, x4 and x5 whichmay be numerical values depending upon the input dataset. It will beunderstood that even though only five inputs are shown in FIG. 105 , invarious implementations, a node may include tens, hundreds, thousands,or more inputs. As discussed above, no computation is performed on theinput layer and thus the outputs from nodes 8842, 8844, 8846, 8848 and8850 of input layer are x1, x2, x3, x4 and x5 respectively, which arefed into hidden layer. The output of node 8852 in the hidden layer maydepend on the outputs from the input layer (x1, x2, x3, x4 and x5) andweights associated with connections (w1, w2, w3, w4 and w5). Thus, theoutput from node 8852 may be computed as:

Y ₈₈₅₂ =f(x1w1+x2w2+x3w3+x4w4+x5w5+b ₈₈₅₂).

The outputs from the nodes 8854 and 8856 in the hidden layer may also becomputed in a similar manner and then be fed to the node 8858 in theoutput layer. Node 8858 in the output layer may perform similarcomputations (using weights v1, v2 and v3 associated with theconnections) as the nodes 8852, 8854 and 8856 in the hidden layers:

Y ₈₈₅₈ =f(y ₈₈₅₂ v1+y ₈₈₅₄ v2+y ₈₈₅₆ v3+b ₈₈₅₈);

where Y₈₈₅₈ is the output of the neural network 8840.

As mentioned, the connections between nodes in the neural network haveassociated weights, which determine how much relative effect an inputvalue has on the output value of the node in question. Before thenetwork is trained, random values are selected for each of the weights.The weights are adjusted during the training process and this adjustmentof weights to determine the best set of weights that maximize theaccuracy of the neural network is referred to as training. For everyinput in a training dataset, the output of the artificial neural networkmay be observed and compared with the expected output, and the errorbetween the expected output and the observed output may be propagatedback to the previous layer. The weights may be adjusted accordinglybased on the error. This process is repeated until the output error isbelow a predetermined threshold.

In embodiments, backpropagation (e.g., backward propagation of errors)is utilized with an optimization method such as gradient descent toadjust weights and update the neural network characteristics.Backpropagation may be a supervised training scheme that learns fromlabeled training data and errors at the nodes by changing parameters ofthe neural network to reduce the errors. For example, a result offorward propagation (e.g., output activation value(s)) determined usingtraining input data is compared against a corresponding known referenceoutput data to calculate a loss function gradient. The gradient may bethen utilized in an optimization method to determine new updated weightsin an attempt to minimize a loss function. For example, to measureerror, the mean square error is determined using the equation:

E=(target−output)²

To determine the gradient for a weight “w,” a partial derivative of theerror with respect to the weight may be determined, where:

gradient=∂E/∂w

The calculation of the partial derivative of the errors with respect tothe weights may flow backwards through the node levels of the neuralnetwork. Then a portion (e.g., ratio, percentage, etc.) of the gradientis subtracted from the weight to determine the updated weight. Theportion may be specified as a learning rate “a.” Thus an exampleequation of determining the updated weight is given by the formula:

w new=w old−α∂E/∂w

The learning rate must be selected such that it is not too small (e.g.,a rate that is too small may lead to a slow convergence to the desiredweights) and not too large (e.g., a rate that is too large may cause theweights to not converge to the desired weights).

After the weight adjustment, the network should perform better thanbefore for the same input because the weights have now been adjusted tominimize the errors.

As mentioned, neural networks may include convolutional neural networks(CNN). A CNN is a specialized neural network for processing data havinga known, grid-like topology, such as image data. Accordingly, CNNs arecommonly used for classification, object recognition and computer visionapplications, but they also may be used for other types of patternrecognition such as speech and language processing.

A convolutional neural network learns highly non-linear mappings byinterconnecting layers of artificial neurons arranged in many differentlayers with activation functions that make the layers dependent. Itincludes one or more convolutional layers, interspersed with one or moresub-sampling layers and non-linear layers, which are typically followedby one or more fully connected layers.

Referring to FIG. 106 , a CNN 8860 includes an input layer with an inputimage 8862 to be classified by the CNN 8860, a hidden layer which inturn includes one or more convolutional layers, interspersed with one ormore activation or non-linear layers (e.g., ReLU) and pooling orsub-sampling layers and an output layer—typically including one or morefully connected layers. Input image 8862 may be represented by a matrixof pixels and may have multiple channels. For example, a colored imagemay have a red, a green, and blue channels each representing red, green,and blue (RGB) components of the input image. Each channel may berepresented by a 2-D matrix of pixels having pixel values in the rangeof 0 to 255. A gray-scale image on the other hand may have only onechannel. The following section describes processing of a single imagechannel using CNN 8860. It will be understood that multiple channels maybe processed in a similar manner.

As shown, input image 8862 may be processed by the hidden layer, whichincludes sets of convolutional and activation layers 8864 and 8868, eachfollowed by pooling layers 8866 and 8870.

The convolutional layers of the convolutional neural network serve asfeature extractors capable of learning and decomposing the input imageinto hierarchical features. The convolution layers may performconvolution operations on the input image where a filter (also referredas a kernel or feature detector) may slide over the input image at acertain step size (referred to as the stride). For every position (orstep), element-wise multiplications between the filter matrix and theoverlapped matrix in the input image may be calculated and summed to geta final value that represents a single element of an output matrixconstituting a feature map. The feature map refers to image data thatrepresents various features of the input image data and may have smallerdimensions as compared to the input image. The activation or non-linearlayers use different non-linear trigger functions to signal distinctidentification of likely features on each hidden layer. Non-linearlayers use a variety of specific functions to implement the non-lineartriggering, including the rectified linear units (ReLUs), hyperbolictangent, absolute of hyperbolic tangent and sigmoid functions. In oneimplementation, a ReLU activation implements the function y=max(x, 0)and keeps the input and output sizes of a layer the same. The advantageof using ReLU is that the convolutional neural network is trained manytimes faster. ReLU is a non-continuous, non-saturating activationfunction that is linear with respect to the input if the input valuesare larger than zero and zero otherwise.

As shown in FIG. 106 , the first convolution and activation layer 8864may perform convolutions on input image 8862 using multiple filtersfollowed by non-linearity operation (e.g., ReLU) to generate multipleoutput matrices (or feature maps) 8872. The number of filters used maybe referred to as the depth of the convolution layer. Thus, the firstconvolution and activation layer 8864 in the example of FIG. 106 has adepth of three and generates three feature maps using three filters.Feature maps 8872 may then be passed to the first pooling layer that maysub-sample or down-sample the feature maps using a pooling function togenerate output matrix 8874. The pooling function replaces the featuremap with a summary statistic to reduce the spatial dimensions of theextracted feature map thereby reducing the number of parameters andcomputations in the network. Thus, the pooling layer reduces thedimensionality of the feature maps while retaining the most importantinformation. The pooling function can also be used to introducetranslation invariance into the neural network, such that smalltranslations to the input do not change the pooled outputs. Differentpooling functions may be used in the pooling layer, including maxpooling, average pooling, and 12-norm pooling.

Output matrix 8874 may then be processed by a second convolution andactivation layer 8868 to perform convolutions and non-linear activationoperations (e.g., ReLU) as described above to generate feature maps8876. In the example shown in FIG. 106 , second convolution andactivation layer 8868 may have a depth of five. Feature maps 8876 maythen be passed to a pooling layer 8870, where feature maps 8876 may besubsampled or down-sampled to generate an output matrix 8878.

Output matrix 8878 generated by pooling layer 8870 is then processed byone or more fully connected layer 8880 that forms a part of the outputlayer of CNN 8860. The fully connected layer 8880 has a full connectionwith all the feature maps of the output matrix 8878 of the pooling layer8870. In embodiments, the fully connected layer 8880 may take the outputmatrix 8878 generated by the pooling layer 8870 as the input in vectorform, and perform high-level determination to output a feature vectorcontaining information of the structures in the input image. Inembodiments, the fully-connected layer 8880 may classify the object ininput image 8862 into one of several categories using a Softmaxfunction. The Softmax function may be used as the activation function inthe output layer and takes a vector of real-valued scores and maps it toa vector of values between zero and one that sum to one. In embodiments,other classifiers, such as a support vector machine (SVM) classifier,may be used.

In embodiments, one or more normalization layers may be added to the CNN8860 to normalize the output of the convolution filters. Thenormalization layer may provide whitening or lateral inhibition, avoidvanishing or exploding gradients, stabilize training, and enablelearning with higher rates and faster convergence. In embodiments, thenormalization layers are added after the convolution layer but beforethe activation layer.

CNN 8860 may thus be seen as multiple sets of convolution, activation,pooling, normalization and fully connected layers stacked together tolearn, enhance and extract implicit features and patterns in the inputimage 8862. A layer as used herein, can refer to one or more componentsthat operate with similar function by mathematical or other functionalmeans to process received inputs to generate/derive outputs for a nextlayer with one or more other components for further processing withinCNN 8860.

The initial layers of CNN 8860 e.g., convolution layers, may extract lowlevel features such as edges and/or gradients from the input image 8862.Subsequent layers may extract or detect progressively more complexfeatures and patterns such as presence of curvatures and textures inimage data and so on. The output of each layer may serve as an input ofa succeeding layer in CNN 8860 to learn hierarchical featurerepresentations from data in the input image 8862. This allowsconvolutional neural networks to efficiently learn increasingly complexand abstract visual concepts.

Although only two convolution layers are shown in the example, thepresent disclosure is not limited to the example architecture, and CNN8860 architecture may comprise any number of layers in total, and anynumber of layers for convolution, activation and pooling. For example,there have been many variations and improvements over the basic CNNmodel described above. Some examples include Alexnet, GoogLeNet, VGGNet(that stacks many layers containing narrow convolutional layers followedby max pooling layers), Residual network or ResNet (that uses residualblocks and skip connections to learn residual mapping), DenseNet (thatconnects each layer of CNN to every other layer in a feed-forwardfashion), Squeeze and excitation networks (that incorporate globalcontext into features) and AmobeaNet (that uses evolutionary algorithmsto search and find optimal architecture for image recognition).

Training of Convolutional Neural Network

The training process of a convolutional neural network, such as CNN8860, may be similar to the training process discussed in FIG. 105 withrespect to neural network 8840.

In embodiments, all parameters and weights (including the weights in thefilters and weights for the fully-connected layer are initially assigned(e.g., randomly assigned). Then, during training, a training image orimages, in which the objects have been detected and classified, areprovided as the input to the CNN 8860, which performs the forwardpropagation steps. In other words, CNN 8860 applies convolution,non-linear activation, and pooling layers to each training image todetermine the classification vectors (i.e., detect and classify eachtraining image). These classification vectors are compared with thepredetermined classification vectors. The error (e.g., the squared sumof differences, log loss, softmax log loss) between the classificationvectors of the CNN and the predetermined classification vectors isdetermined. This error is then employed to update the weights andparameters of the CNN in a backpropagation process which may usegradient descent and may include one or more iterations. The trainingprocess is repeated for each training image in the training set.

The training process and inference process described above may beperformed on hardware, software, or a combination of hardware andsoftware. However, training a convolutional neural network like CNN 8860or using the trained CNN for inference generally requires significantamounts of computation power to perform, for example, the matrixmultiplications or convolutions. Thus, specialized hardware circuits,such as graphic processing units (GPUs), tensor processing units (TPUs),neural network processing units (NPUs), FPGAs, ASICs, or other highlyparallel processing circuits may be used for training and/or inference.Training and inference may be performed on a cloud, on a data center, oron a device.

Region Based CNNS (RCNNS) and Object Detection

In embodiments, an object detection model extends the functionality ofCNN based image classification neural network models by not onlyclassifying objects but also determining their locations in an image interms of bounding boxes. Region-based CNN (R-CNN) methods are used toextract regions of interest (ROI), where each ROI is a rectangle thatmay represent the boundary of an object in image. Conceptually, R-CNNoperates in two phases. In a first phase, region proposal methodsgenerate all potential bounding box candidates in the image. In a secondphase, for every proposal, a CNN classifier is applied to distinguishbetween objects. Alternatively, a fast R-CNN architecture can be used,which integrates the feature extractor and classifier into a unifiednetwork. Another faster R-CNN can be used, which incorporates a RegionProposal Network (RPN) and fast R-CNN into an end-to-end trainableframework. Mask R-CNN adds instance segmentation, while mesh R-CNN addsthe ability to generate a 3D mesh from a 2D image.

In embodiments, artificial intelligence modules 8804 may provide accessto and/or integrate a robotic process automation (RPA) module 8816. TheRPA module 8816 may facilitate, among other things, computer automationof producing and validating workflows. In embodiments, an RPA module8816 may monitor human interaction with various systems to learnpatterns and processes performed by humans in performance of respectivetasks. This may include observation of human actions that involveinteractions with hardware elements, with software interfaces, and withother elements. Observations may include field observations as humansperform real tasks, as well as observations of simulations or otheractivities in which a human performs an action with the explicit intentto provide a training data set or input for the RPA system, such aswhere a human tags or labels a training data set with features thatassist the RPA system in learning to recognize or classify features orobjects, among many other examples. In embodiments, an RPA module 8816may learn to perform certain tasks based on the learned patterns andprocesses, such that the tasks may be performed by the RPA module 8816in lieu or in support of a human decision maker. Examples of RPA modules8816 may encompass those in this disclosure and in the documentsincorporated by reference herein and may involve automation of any ofthe wide range of value chain network activities or entities describedtherein.

In embodiments, the artificial intelligence modules 8804 may includeand/or provide access to an analytics module 8818. In embodiments, ananalytics module 8818 is configured to perform various analyticalprocesses on data output from value chain entities or other datasources. In example embodiments, analytics produced by the analyticsmodule 8818 may facilitate quantification of system performance ascompared to a set of goals and/or metrics. The goals and/or metrics maybe preconfigured, determined dynamically from operating results, and thelike. Examples of analytics processes that can be performed by ananalytics module 8818 are discussed below and in the documentincorporated herein by reference. In some example implementations,analytics processes may include tracking goals and/or specific metricsthat involve coordination of value chain activities and demandintelligence, such as involving forecasting demand for a set of relevantitems by location and time (among many others).

In embodiments, artificial intelligence modules 8804 may include and/orprovide access to a digital twin module 8820. The digital twin module8820 may encompass any of a wide range of features and capabilitiesdescribed herein In embodiments, a digital twin module 8820 may beconfigured to provide, among other things, execution environments forand different types of digital twins, such as twins of physicalenvironments, twins of robot operating units, logistics twins, executivedigital twins, organizational digital twins, role-based digital twins,and the like. In embodiments, the digital twin module 8820 may beconfigured in accordance with digital twin systems and/or modulesdescribed elsewhere throughout the disclosure. In example embodiments, adigital twin module 8820 may be configured to generate digital twinsthat are requested by intelligence clients 8836. Further, the digitaltwin module 8820 may be configured with interfaces, such as APIs and thelike for receiving information from external data sources. For instance,the digital twin module 8820 may receive real-time data from sensorsystems of a machinery, vehicle, robot, or other device, and/or sensorsystems of the physical environment in which a device operates. Inembodiments, the digital twin module 8820 may receive digital twin datafrom other suitable data sources, such as third-party services (e.g.,weather services, traffic data services, logistics systems anddatabases, and the like. In embodiments, the digital twin module 8820may include digital twin data representing features, states, or the likeof value chain network entities, such as supply chain infrastructureentities, transportation or logistic entities, containers, goods, or thelike, as well as demand entities, such as customers, merchants, stores,points-of-sale, points-of-use, and the like. The digital twin module8820 may be integrated with or into, link to, or otherwise interact withan interface (e.g., a control tower or dashboard), for coordination ofsupply and demand, including coordination of automation within supplychain activities and demand management activities.

In embodiments, a digital twin module 8820 may provide access to andmanage a library of digital twins. Artificial intelligence modules 8804may access the library to perform functions, such as a simulation ofactions in a given environment in response to certain stimuli.

In embodiments, artificial intelligence modules 8804 may include and/orprovide access to a machine vision module 8822. In embodiments, amachine vision module 8822 is configured to process images (e.g.,captured by a camera) to detect and classify objects in the image. Inembodiments, the machine vision module 8822 receives one or more images(which may be frames of a video feed or single still shot images) andidentifies “blobs” in an image (e.g., using edge detection techniques orthe like). The machine vision module 8822 may then classify the blobs.In some embodiments, the machine vision module 8822 leverages one ormore machine-learned image classification models and/or neural networks(e.g., convolutional neural networks) to classify the blobs in theimage. In some embodiments, the machine vision module 8822 may performfeature extraction on the images and/or the respective blobs in theimage prior to classification. In some embodiments, the machine visionmodule 8822 may leverage classification made in a previous image toaffirm or update classification(s) from the previous image. For example,if an object that was detected in a previous frame was classified with alower confidence score (e.g., the object was partially occluded or outof focus), the machine vision module 8822 may affirm or update theclassification if the machine vision module 8822 is able to determine aclassification of the object with a higher degree of confidence. Inembodiments, the machine vision module 8822 is configured to detectocclusions, such as objects that may be occluded by another object. Inembodiments, the machine vision module 8822 receives additional input toassist in image classification tasks, such as from a radar, a sonar, adigital twin of an environment (which may show locations of knownobjects), and/or the like. In some embodiments, a machine-vision module8822 may include or interface with a liquid lens. In these embodiments,the liquid lens may facilitate improved machine vision (e.g., whenfocusing at multiple distances is necessitated by the environment andjob of a robot) and/or other machine vision tasks that are enabled by aliquid lens.

In embodiments, the artificial intelligence modules 8804 may includeand/or provide access to a natural language processing (NLP) module8824. In embodiments, an NLP module 8824 performs natural language taskson behalf of an intelligence service client 8836. Examples of naturallanguage processing techniques may include, but are not limited to,speech recognition, speech segmentation, speaker diarization,text-to-speech, lemmatization, morphological segmentation,parts-of-speech tagging, stemming, syntactic analysis, lexical analysis,and the like. In embodiments, the NLP module 8824 may enable voicecommands that are received from a human. In embodiments, the NLP module8824 receives an audio stream (e.g., from a microphone) and may performvoice-to-text conversion on the audio stream to obtain a transcriptionof the audio stream. The NLP module 8824 may process text (e.g., atranscription of the audio stream) to determine a meaning of the textusing various NLP techniques (e.g., NLP models, neural networks, and/orthe like). In embodiments, the NLP module 8824 may determine an actionor command that was spoken in the audio stream based on the results ofthe NLP. In embodiments, the NLP module 8824 may output the results ofthe NLP to an intelligence service client 8836.

In embodiments, the NLP module 8824 provides an intelligence serviceclient 8836 with the ability to parse one or more conversational voiceinstructions provided by a human user to perform one or more tasks aswell as communicate with the human user. The NLP module 8824 may performspeech recognition to recognize the voice instructions, natural languageunderstanding to parse and derive meaning from the instructions, andnatural language generation to generate a voice response for the userupon processing of the user instructions. In some embodiments, the NLPmodule 8824 enables an intelligence service client 8836 to understandthe instructions and, upon successful completion of the task by theintelligence service client 8836, provide a response to the user. Inembodiments, the NLP module 8824 may formulate and ask questions to auser if the context of the user request is not completely clear. Inembodiments, the NLP module 8824 may utilize inputs received from one ormore sensors including vision sensors, location-based data (e.g., GPSdata) to determine context information associated with processed speechor text data.

In embodiments, the NLP module 8824 uses neural networks when performingNLP tasks, such as recurrent neural networks, long short term memory(LSTMs), gated recurrent unit (GRUs), transformer neural networks,convolutional neural networks and/or the like.

FIG. 107 illustrates an example neural network 8800 for implementing NLPmodule 8824. In the illustrated example, the example neural network is atransformer neural network. In the example, the transformer neuralnetwork 8800 includes three input stages and five output stages totransform an input sequence into an output sequence. The exampletransformer includes an encoder 8802 and a decoder 8804. The encoder8802 processes input, and the decoder 8804 generates outputprobabilities, for example. The encoder 8802 includes three stages, andthe decoder 8804 includes five stages. Encoder 8802 stage 1 representsan input as a sequence of positional encodings added to embedded inputs.Encoder 8802 stages 2 and 3 include N layers (e.g., N=6, etc.) in whicheach layer includes a position-wise feedforward neural network (FNN) andan attention-based sublayer. Each attention-based sublayer of encoder8802 stage 2 includes four linear projections and multi-head attentionlogic to be added and normalized to be provided to the position-wise FNNof encoder 8802 stage 3. Encoder 8802 stages 2 and 3 employ a residualconnection followed by a normalization layer at their output.

The example decoder 8804 processes an output embedding as its input withthe output embedding shifted right by one position to help ensure that aprediction for position i is dependent on positions previous to/lessthan i. In stage 2 of the decoder 8804, masked multi-head attention ismodified to prevent positions from attending to subsequent positions.Stages 3-4 of the decoder 8804 include N layers (e.g., N=6, etc.) inwhich each layer includes a position-wise FNN and two attention-basedsublayers. Each attention-based sublayer of decoder 8804 stage 3includes four linear projections and multi-head attention logic to beadded and normalized to be provided to the position-wise FNN of decoder8804 stage 4. Decoder 8804 stages 2-4 employ a residual connectionfollowed by a normalization layer at their output. Decoder 8804 stage 5provides a linear transformation followed by a softmax function tonormalize a resulting vector of K numbers into a probabilitydistribution 8806 including K probabilities proportional to exponentialsof the K input numbers.

Additional examples of neural networks may be found elsewhere in thedisclosure (e.g., FIGS. 78-103 ).

Referring back to FIG. 104 , in embodiments, artificial intelligencemodules 8804 may also include and/or provide access to a rules-basedmodule 8828 that may be integrated into or be accessed by anintelligence service client 8836. In some embodiments, a rules-basedmodule 8828 may be configured with programmatic logic that defines a setof rules and other conditions that trigger certain actions that may beperformed in connection with an intelligence client. In embodiments, therule-based module 8828 may be configured with programmatic logic thatreceives input and determines whether one or more rules are met based onthe input. If a condition is met, the rules-based module 8828 determinesan action to perform, which may be output to a requesting intelligenceservice client 8836. The data received by the rules-based engine may bereceived from an intelligence service input source 8832 and/or may berequested from another module in artificial intelligence modules 8804,such as the machine vision module 8822, the neural network module 8814,the ML module 8812, and/or the like. For example, a rule-based module8828 may receive classifications of objects in a field of view of amobile system (e.g., robot, autonomous vehicle, or the like) from amachine vision system and/or sensor data from a lidar sensor of themobile system and, in response, may determine whether the mobile systemshould continue in its path, change its course, or stop. In embodiments,the rules-based module 8828 may be configured to make other suitablerules-based decisions on behalf of a respective client 8836, examples ofwhich are discussed throughout the disclosure. In some embodiments, therules-based engine may apply governance standards and/or analysismodules, which are described in greater detail below.

In embodiments, artificial intelligence modules 8804 interface with anintelligence service controller 8802, which is configured to determine atype of request issued by an intelligence service client 8836 and, inresponse, may determine a set of governance standards and/or analysesthat are to be applied by the artificial intelligence modules 8804 whenresponding to the request. In embodiments, the intelligence servicecontroller 8802 may include an analysis management module 8806, a set ofanalysis modules 8808, and a governance library 8810.

In embodiments, an intelligence service controller 8802 is configured todetermine a type of request issued by an intelligence service client8836 and, in response, may determine a set of governance standardsand/or analyses that are to be applied by the artificial intelligencemodules 8804 when responding to the request. In embodiments, theintelligence service controller 8802 may include an analysis managementmodule 8806, a set of analysis modules 8808, and a governance library8810. In embodiments, the analysis management module 8806 receives anartificial intelligence module 8804 request and determines thegovernance standards and/or analyses implicated by the request. Inembodiments, the analysis management module 8806 may determine thegovernance standards that apply to the request based on the type ofdecision that was requested and/or whether certain analyses are to beperformed with respect to the requested decision. For example, a requestfor a control decision that results in an intelligence service client8836 performing an action may implicate a certain set of governancestandards that apply, such as safety standards, legal standards, qualitystandards, or the like, and/or may implicate one or more analysesregarding the control decision, such as a risk analysis, a safetyanalysis, an engineering analysis, or the like.

In some embodiments, the analysis management module 8806 may determinethe governance standards that apply to a decision request based on oneor more conditions. Non-limiting examples of such conditions may includethe type of decision that is requested, a geolocation in which adecision is being made, an environment that the decision will affect,current or predicted environment conditions of the environment and/orthe like. In embodiments, the governance standards may be defined as aset of standards libraries stored in a governance library 8810. Inembodiments, standards libraries may define conditions, thresholds,rules, recommendations, or other suitable parameters by which a decisionmay be analyzed. Examples of standards libraries may include, legalstandards library, a regulatory standards library, a quality standardslibrary, an engineering standards library, a safety standards library, afinancial standards library, and/or other suitable types of standardslibraries. In embodiments, the governance library 8810 may include anindex that indexes certain standards defined in the respective standardslibrary based on different conditions. Examples of conditions may be ajurisdiction or geographic areas to which certain standards apply,environmental conditions to which certain standards apply, device typesto which certain standards apply, materials or products to which certainstandards apply, and/or the like.

In some embodiments, the analysis management module 8806 may determinethe appropriate set of standards that must be applied with respect to aparticular decision and may provide the appropriate set of standards tothe artificial intelligence modules 8804, such that the artificialintelligence modules 8804 leverages the implicated governance standardswhen determining a decision. In these embodiments, the artificialintelligence modules 8804 may be configured to apply the standards inthe decision-making process, such that a decision output by theartificial intelligence modules 8804 is consistent with the implicatedgovernance standards. It is appreciated that the standards libraries inthe governance library may be defined by the platform provider,customers, and/or third parties. The standards may be governmentstandards, industry standards, customer standards, or other suitablesources. In embodiments, each set of standards may include a set ofconditions that implicate the respective set of standards, such that theconditions may be used to determine which standards to apply given asituation.

In some embodiments, the analysis management module 8806 may determineone or more analyses that are to be performed with respect to aparticular decision and may provide corresponding analysis modules 8808that perform those analyses to the artificial intelligence modules 8804,such that the artificial intelligence modules 8804 leverage thecorresponding analysis modules 8808 to analyze a decision beforeoutputting the decision to the requesting client. In embodiments, theanalysis modules 8808 may include modules that are configured to performspecific analyses with respect to certain types of decisions, wherebythe respective modules are executed by a processing system that hoststhe instance of the intelligence services 8800. Non-limiting examples ofanalysis modules 8808 may include risk analysis module(s), securityanalysis module(s), decision tree analysis module(s), ethics analysismodule(s), failure mode and effects (FMEA) analysis module(s), hazardanalysis module(s), quality analysis module(s), safety analysismodule(s), regulatory analysis module(s), legal analysis module(s),and/or other suitable analysis modules.

In some embodiments, the analysis management module 8806 is configuredto determine which types of analyses to perform based on the type ofdecision that was requested by an intelligence service client 8836. Insome of these embodiments, the analysis management module 8806 mayinclude an index or other suitable mechanism that identifies a set ofanalysis modules 8808 based on a requested decision type. In theseembodiments, the analysis management module 8806 may receive thedecision type and may determine a set of analysis modules 8808 that areto be executed based on the decision type. Additionally oralternatively, one or more governance standards may define when aparticular analysis is to be performed. For example, the engineeringstandards may define what scenarios necessitate a FMEA analysis. In thisexample, the engineering standards may have been implicated by a requestfor a particular type of decision and the engineering standards maydefine scenarios when an FMEA analysis is to be performed. In thisexample, artificial intelligence modules 8804 may execute a safetyanalysis module and/or a risk analysis module and may determine analternative decision if the action would violate a legal standard or asafety standard. In response to analyzing a proposed decision,artificial intelligence modules 8804 may selectively output the proposedcondition based on the results of the executed analyses. If a decisionis allowed, artificial intelligence modules 8804 may output the decisionto the requesting intelligence service client 8836. If the proposedconfiguration is flagged by one or more of the analyses, artificialintelligence modules 8804 may determine an alternative decision andexecute the analyses with respect to the alternate proposed decisionuntil a conforming decision is obtained.

It is noted here that in some embodiments, one or more analysis modules8808 may themselves be defined in a standard, and one or more relevantstandards used together may comprise a particular analysis. For example,the applicable safety standard may call for a risk analysis that can useor more allowable methods. In this example, an ISO standard for overallprocess and documentation, and an ASTM standard for a narrowly definedprocedure may be employed to complete the risk analysis required by thesafety governance standard.

As mentioned, the foregoing framework of an intelligence services 8800may be applied in and/or leveraged by various entities of a value chain.For example, in some embodiments, a platform-level intelligence systemmay be configured with the entire capabilities of the intelligenceservices 8800, and certain configurations of the intelligence services8800 may be provisioned for respective value chain entities.Furthermore, in some embodiments, an intelligence service client 8836may be configured to escalate an intelligence system task to ahigher-level value chain entity (e.g., edge-level or the platform-level)when the intelligence service client 8836 cannot perform the taskautonomously. It is noted that in some embodiments, an intelligenceservice controller 8802 may direct intelligence tasks to a lower-levelcomponent. Furthermore, in some implementations, an intelligenceservices 8800 may be configured to output default actions when adecision cannot be reached by the intelligence services 8800 and/or ahigher or lower-level intelligence system. In some of theseimplementations, the default decisions may be defined in a rule and/orin a standards library.

Reinforcement Learning to Determine Optimal Policy

Reinforcement learning (RL), is a machine learning technique where anagent iteratively learns optimal policy through interactions with theenvironment. In RL, the agent must discover correct actions bytrial-and-error so as to maximize some notion of long-term reward.Specifically, in a system employing RL, there exist two entities: (1) anenvironment and (2) an agent. The agent is a computer program componentthat is connected to its environment such that it can sense the state ofthe environment as well as execute actions on the environment. On eachstep of interaction, the agent senses the current state of theenvironment, s, and chooses an action to take, a. The action changes thestate of the environment, and the value of this state transition iscommunicated to the agent by a reward signal, r, where the magnitude ofr indicates the desirability of an action. Over time, the agent builds apolicy, π, which specifies the action the agent will take for each stateof the environment.

Formally, in reinforcement learning, there exists a discrete set ofenvironment states, S; a discrete set of agent actions, A; and a set ofscalar reinforcement signals, R. After learning, the system creates apolicy, π, that defines the value of taking action aεA in state sεS. Thepolicy defines Qπ(s, a) as the expected return value for starting froms, taking action a, and following policy π.

The reinforcement learning agent is trained in a policy throughiterative exposure to various states, having the agent select an actionas per the policy and providing a reward based on a function designed toreward desirable behavior. Based on the reward feedback, the system may“learn” the policy and becomes trained in producing desirable actions.For example, for navigation policy, RL agent may evaluate its staterepeatedly (e.g., location, distance from a target object), select anaction (e.g., provide input to the motors for movement towards thetarget object), evaluate the action using a reward signal, whichprovides an indication of the of the success of the action. (e.g., areward of +10 if movement reduces the distance between a mobile systemand a target object and −10 if the movement increases the distance).Similarly, the RL agent may be trained in grasping policy by iterativelyobtaining images of a target object to be grasped, attempt to grasp theobject, evaluate the attempt, and then execute the subsequent iterationusing the evaluation of the attempt of the preceding iteration(s) toassist in determining the next attempt.

There may be several approaches for training the RL agent in a policy.Imitation learning is a key approach in which the agent learns fromstate/action pairs where the actions are those that would be chosen byan expert (e.g., a human) in response to an observed state. Imitationlearning not just solves sample-inefficiency or computationalfeasibility problems, but also makes the training process safer. The RLagent may derive multiple examples of the state/action pairs byobserving a human (e.g., navigating towards and grasping a targetobject), and uses them as a basis for training the policy. Behaviorcloning (BC), that focuses on learning the expert's policy usingsupervised learning is an example of imitation learning approach.

Value based learning approach aims to find a policy comprising asequence of actions that maximizes the expectation value of futurereward (or minimizes the expected cost). The RL agent may learn thevalue/cost function and then derives a policy with respect to the same.Two different expectation values are often referred to: the state valueV(s) and the action value Q (s,a) respectively. The state value functionV(s) represents the value associated with the agent at each statewhereas the action value function Q(s,a) represents the value associatedwith the agent at state s and performing action a. The value-basedlearning approach works by approximating optimal value (V* or Q*) andthen deriving an optimal policy. For example, the optimal value functionQ*(s, a) may be identified by finding the sequence of actions whichmaximize the state-action value function Q (s, a). The optimal policyfor each state can be derived by identifying the highest valued actionthat can be taken from each state.

π*(s)=argmax Q*(s,a)

To iteratively calculate the value function as actions within thesequence are executed and the mobile system transitions from one stateto another, the Bellman Optimality equation may be applied. The optimalvalue function Q*(s,a) obeys Bellman Optimality equation and can beexpressed as:

Q*(s _(t) ,a _(t))=E[r _(t+1)+γmax Q*(s _(t+1) ,a _(t+1))]

Policy based learning approach directly optimizes the policy function πusing a suitable optimization technique (e.g., stochastic gradientdescent) to fine tune a vector of parameters without calculating a valuefunction. The policy-based learning approach is typically effective inhigh-dimensional or continuous action spaces.

FIG. 108 illustrates an approach based on reinforcement learning andincluding evaluation of various states, actions and rewards indetermining optimal policy for executing one or more tasks by a mobilesystem.

At 8902, a reinforcement learning agent (e.g., of the intelligenceservices system 8900) receives sensor information including a pluralityof images captured by the mobile system in the environment. The analysisof one or more of these images may enable the agent to determine a firststate associated with the mobile system at 8904. The data representingthe first state may include information about the environment, such asimages, sounds, temperature or time and information about the mobilesystem, including its position, speed, internal state (e.g., batterylife, clock setting) etc.

At 8906, 8908, and 8910, various potential actions responsive to thestate may be determined. Some examples of potential actions includeproviding control instructions to actuators, motors, wheels, wingsflaps, or other components that controls the agent's speed,acceleration, orientation, or position; changing the agent's internalsettings, such as putting certain components into a sleep mode toconserve battery life; changing the direction if the agent is in dangerof colliding with an obstacle object; acquiring or transmitting data;attempting to grasp a target object and the like.

At 8912, 8914 and 8916, expected rewards may be determined for each ofthe potential actions based on a reward function. For each of thedetermined potential actions, an expected reward may be determined basedon a reward function. The reward may be predicated on a desired outcome,such as avoiding an obstacle, conserving power, or acquiring data. Ifthe action yields the desired outcome (e.g., avoiding the obstacle), thereward is high; otherwise, the reward may be low.

The agent may also look to the future to analyze whether there may beopportunities for realizing higher rewards in the future. At 8918, 8920,and 8922, the agent may determine future states resulting from potentialactions respectively at 8906, 8908, and 8910.

For each of the future states predicted at 8918, 8920, and 8922, one ormore future actions may be determined and evaluated. At steps 8924,8926, and 8928, for example, values or other indicators of expectedrewards associated with one or more of the future actions may bedeveloped. The expected rewards associated with the one or more futureactions may be evaluated by comparing values of reward functionsassociated with each future action

At 8930, an action may be selected based on a comparison of expectedcurrent and future rewards.

In embodiments, the reinforcement learning agent may be pre-trainedthrough simulations in a digital twin system. In embodiments, thereinforcement agent may be pre-trained using behavior cloning. Inembodiments, the reinforcement agent may be trained using a deepreinforcement learning algorithm selected from Deep Q-Network (DQN),double deep Q-Network (DDQN), Deep Deterministic Policy Gradient (DDPG),soft actor critic (SAC), advantage actor critic (A2C), asynchronousadvantage actor critic (A3C), proximal policy optimization (PPO), trustregion policy optimization (TRPO).

In embodiments, the reinforcement learning agent may look to balanceexploitation (of current knowledge) with exploration (of unchartedterritory) while traversing the action space. For example, the agent mayfollow an ε-greedy policy by randomly selecting exploration occasionallywith probability c while taking the optimal action most of the time withprobability 1-ε, where ε is a parameter satisfying 0<ε<1.

Specialized Chips

FIGS. 109-113 illustrate a plurality of specialized chips that providevarious system functionalities for use in a variety of contexts, andthat may be leveraged in systems described herein and/or to providefunctionalities described herein. As explained in more detail below, thechip functionalities are configurable for specific contexts and toaddress specific tasks. Therefore, using the functionalities of one ormore of the chips, systems of systems such as those described herein maybe more easily created, configured, deployed, and reconfigured. Any ofthe chips may be used in the various systems described herein and byvarious value chain entities in ways that will be evident from thedisclosures of the capabilities of each chip.

FIG. 109 illustrates a physical orientation determination chip 9100, oneor more of which may be used to determine data about one or morephysical orientations as described herein. The chip 9100 may be used byany value chain entity that leverages mobile systems. In embodiments,the chip(s) 9100 may use artificial intelligence (AI) and othertechniques to determine the physical orientation of a mobile system. Asdescribed herein, the chip(s) 9100 may receive one or more inputs 9192from a mobile system and perform one or more AI-assisted functions todetermine the physical orientation of the mobile system. The chip(s)9100 may then transmit outputs 9194 indicating the determined physicalorientation. The chip(s) 9100 may be part of a mobile system (e.g., arobot), and/or may be part of a different device (e.g., a base stationin communication with the robot) that receives inputs 9192 from themobile system. A mobile system may include any system that is mobileand/or that has one or more mobile components as described herein.

The physical orientation(s) determined by the chip(s) 9100 may berelative to any real reference point/frame (e.g., the solar system, GPScoordinates, coordinates within another system, etc.) or simulatedreference point/frame (e.g., coordinates with an environment digitaltwin or other virtual space). In embodiments, the physical orientationmay include a location, a rotation/heading (e.g., a direction the mobilesystem is facing towards and/or angle at which the mobile system isrotated), a tilt (e.g., an amount the mobile system is leaning in one ormore directions), velocity, and/or acceleration, each of which may berelative to any real or simulated point/frame. Accordingly, theoutput(s) 9194 may comprise one or more data structures indicating thevarious orientation information.

In embodiments, the chip(s) 9100 may determine and/or output theorientation of the entire mobile system. Additionally or alternatively,the chip(s) 9100 may determine and/or output the orientation of one ormore components (e.g., limbs, wheels, instruments, appendages, or othercomponents) of the mobile system.

In embodiments, the chip(s) 9100 can be modular component(s) that may beintegrated with the mobile system in various ways. As stated above, thechip(s) may be integrated with a mobile system and/or integrated with asystem in communication with the mobile system. To facilitate thismodularity, the chip(s) 9100 may be provided partially or completelywithin a housing (not shown) and may receive the inputs 9192 and/orprovide the outputs 9194 via electrical connectors, optical connectors,and/or wireless connectors (e.g., antennae, inductive coils, etc.).Additionally or alternatively, the chip(s) 9100 may be integrated withother circuits, processors, systems, etc., either on one or multiplesubstrates/chips.

The chip(s) 9100 may be and/or include one or more system-on-chips(SOCs), integrated circuits (ICs), application-specific integratedcircuits (ASICs), and/or the like, for providing the functionalityattributed to chip 9100 and/or any other functionality. For example, thechip 9100 may be provided as part of a SOC that also provides otherfunctions described herein. In general, the components of the chip 9100may comprise one or more general-purpose processing chips that areconfigured using software instructions or other code, and/or maycomprise special-purpose processing chips (e.g., ASICs) customized toperform the functions described herein.

Multiple chip(s) 9100 may be used to perform the functions describedherein. For example, multiple chip(s) 9100 may use serial, parallel,and/or other processing techniques to determine physical orientationdata more quickly, to determine physical orientation data moreefficiently by offloading more complex computations from one chip 9100to another chip 9100 with a better power source, and/or the like. Asanother example, one chip 9100 may be used to provide physicalorientation data for one component of the mobile system (e.g., a leftarm/leg/wheel), while another chip 9100 may be used to provide physicalorientation data for a second component of the mobile system (e.g., aright arm/leg/wheel).

In embodiments, the physical input interface 9102 receives one or moreinputs 9192 to the physical orientation determination chip 9100 asdescribed herein. The inputs 9192 may be transmitted to the physicalinput interface 9102 by other chips, circuits, modules, and/or othercomponents of the mobile system. For example, the input data may comefrom sensors, sensor-processing chips/modules/circuits, antennae,storage devices, network interfaces, or any other source of data for thechip(s) 9100 as described herein. The physical input interface 9102 mayconnect with the source(s) of the inputs 9192 via wired or wirelessconnections. The inputs 9192 may include one or more of locationsignals/data, accelerometer, gyroscope, or other relative motion data,image, video, or other vision data, as well as LIDAR data, radar data,sonar data, and/or the like. The inputs 9192 may also include data thatmay be stored in storage 9150, such as images for image library 9152,data for an environment digital twin 9154 (e.g., a digitalrepresentation of the environment surrounding the mobile system), one ormore system specification(s) 9156, and/or one or more intelligencemodule(s) 9158.

As stated above, the output data 9194 transmitted from the physicaloutput interface 9104 may include one or more of data indicating thelocation, rotation/heading, tilt, velocity, and/or acceleration asdetermined by the chip 9100. In embodiments, the outputs of the chip9100 may be transmitted by the physical output interface 9104 to otherchips, circuits, modules, and/or other components as described herein.The physical output interface 9104 may connect to these components viawired or wireless connections.

In embodiments, the chip 9100 may include one or more of a locationmodule 9110, a relative motion module 9120, a machine vision module9130, and an orientation module 9140. In embodiments, the locationmodule 9110 may comprise circuits 9112-9116 for determining andoutputting a location (e.g., GPS coordinates) based on the inputs 9192.Additionally or alternatively, the chip 9100 may include a relativemotion module 9120 comprising circuits 9122-9126 for determining andoutputting a relative motion (e.g., a change inposition/rotation/heading, velocity information, and/or accelerationinformation) based on the inputs 9192. Additionally or alternatively,the chip 9100 may include a machine vision module 9130 comprisingcircuits 9132-9136 for analyzing image data provided as inputs 9192 todetect and/or classify objects. Additionally or alternatively, the chip9100 may include an orientation module 9140 comprising circuits9142-9148 for generating an environment digital twin (e.g., a digitalrepresentation of an environment), retrieving a stored environmentdigital twin, and/or updating an environment digital twin, determining alocation of the mobile system (e.g., a location within an environment orenvironment digital twin), determining a pose of the mobile system(e.g., an arrangement of one or more wheels, limbs, instruments,appendages, or other mobile system components), and determiningorientation information for transmitting as outputs 9194. Thefunctionalities of the various circuits of the modules 9110, 9120, 9130,and/or 9140 are described in more detail below.

The processing core(s) 9106 may comprise one or more processing core(s)that may be configured to perform any of the functions attributed to thechip 9100, either with or without the assistance of the various modules9110, 9120, 9130, and/or 9140. For example, the processing core(s) 9106may leverage and/or invoke various modules to perform various functionsdescribed herein. The processing core(s) 9106 may comprisegeneral-purpose and/or special-purpose processors. In embodiments, theprocessing core(s) 9106 may use serial, parallel, and/or otherprocessing techniques to accomplish the functions described herein.

Accordingly, the processing core(s) 9106 may perform functions inaddition to the functions provided by the various modules 9110, 9120,9130, and/or 9140. For example, the processing core(s) may receive anoutput of one module (e.g., a location output by location module 9110)and provide it as input to another module (e.g., to the orientationmodule 9140). The processing core(s) 9106 may also process the output ofany of the module(s) to convert the output into a different format.

The processing core(s) 9106 may also compare the data output bydifferent modules for error checking and/or to enhance accuracy. Forexample, if the location module 9110 indicates that a location of asystem has changed, but the relative motion module 9120 indicates thatthe system's location has not changed (e.g., a location signal may beincorrect due to a reflected signal or due to the imprecision of GPS atgranular levels), the processing core(s) 9106 may discard and/or modifythe output of the location module 9110.

In embodiments, the processing core(s) 9106 may generate data based onthe outputs of different modules. For example, the processing core(s)9106 may determine a velocity vector data structure based on both acurrent location output by the location module 9110 and on the relativemotion output by the relative motion module 9120. Other outputs ofvarious modules may be combined in similar ways.

In embodiments, the processing core(s) 9106 may further operate to storeand/or retrieve data to/from storage 9150. For example, the processingcore(s) 9106 may store and retrieve images in an image library 9152(e.g., for use by the machine vision module 9130, as described in moredetail below), may store and retrieve an environment digital twin 9154(e.g., as generated/updated by the orientation module 9140, as describedin more detail below), may store and retrieve system specification(s)9156 (e.g., for determining information about components of the mobilesystem), and/or may store and retrieve intelligence module(s) 9158 forimplementing the various functions described herein. In embodiments, theprocessing core(s) may implement any of the functionalities of theintelligence service 8800 (as described with respect to FIG. 104 ) usingthe intelligence modules 9158 (which may include one or more of theartificial intelligence modules 8804 of FIG. 104 ).

The location module 9110 may receive location signals (e.g., GPSsignals, cellular signals, WI-FI signals) and determine a location(e.g., GPS coordinates or coordinates within some other real orsimulated coordinate system/frame). In some embodiments, the locationsignal capture circuit 9112 may receive location signal data from theinputs 9192 and perform initial processing on the location signal datato capture data from the location signal (e.g., demodulation, storage ina buffer, initial sanity checking, etc.). In some cases (e.g., if thelocation is being determined within coordinates of an environmentdigital twin), the location signal capture circuit 9112 may retrieve anenvironment digital twin 9154 from storage and/or from an environmentdigital twin circuit 9142. The location determination circuit 9114 maythen calculate a location based on the captured location data. Forexample, the location determination circuit 9114 may use trilaterationtechniques to compute GPS coordinates and related data (e.g.,accuracy/error data) based on GPS signals received from multiplesatellites. As another example, the location determination circuit 9114may use cellular and/or WI-FI data to determine a location of the mobilesystem. In embodiments, multiple location signals may be used by thelocation determination circuit 9114 to improve accuracy. The locationoutput circuit 9116 may then output (e.g., to the processing core(s)9106) the location data (e.g., one or more data structures indicatingcoordinates and/or related data), which in turn may provide the locationdata to other modules, output the location data as outputs 9194, orotherwise process the location data to determine orientationinformation.

The relative motion module 9120 may receive accelerometer, gyroscope,and/or other relative motion signals as inputs 9192 and determinerelative motion data (e.g., change in position and/or rotation/heading,velocity data, and/or acceleration data) with respect to one or morereal or simulated points/frames. The motion sensor capture circuit 9122may receive data signals from motion sensors such as accelerometers,gyroscopes, and the like and perform initial processing on the data tocapture the relative motion data (e.g., demodulation, storage in abuffer, initial sanity checking, etc.). In some cases (e.g., if therelative motion is being determined with respect to an environmentdigital twin), the motion sensor capture circuit 9122 may retrieve anenvironment digital twin 9154 from storage and/or from an environmentdigital twin circuit 9142. The relative motion determination circuit9124 may then process the relative motion data using integrationtechniques, dead reckoning techniques, and/or the like to generaterelative motion data (e.g., one or more data structures indicatingchange in position/rotation/heading, velocity, angular velocity,acceleration, angular acceleration, and/or the like) with respect to agiven point/frame, whether real or simulated. The relative motion outputcircuit 9126 may then output (e.g., to the processing core(s) 9106) therelative motion data, which in turn may provide the relative motion datato other modules, output the relative motion data as outputs 9194, orotherwise process the relative motion data to determine orientationinformation.

In embodiments, the machine vision module 9130 may receive image, video,or other vision-related signals (e.g., LIDAR data) and process the datato detect and/or classify objects. The image sensor capture circuit 9132may receive vision-related signals from the inputs 9192 and performinitial processing on the vision-related signals to capture images orother vision data (e.g., demodulation, storage in a buffer, extractionof images from video, image generation based on LIDAR data, etc.). Theobject detection circuit 9134 may then detect one or more objectsappearing in the image or other vision data. For example, the objectdetection circuit 9134 may use image-processing techniques such asline/edge detection and/or other machine-learning techniques to detectthe location of objects in image/vision data. In some embodiments, theobject detection circuit 9134 may leverage machine-learned models (e.g.,stored as intelligence modules 9158) for object detection.

The object classification circuit 9136 may recognize or otherwiseclassify objects appearing in the image or other vision data. In someembodiments (not shown), the object detection circuit 9134 and theobject classification circuit 9136 may be the same circuit. For example,the machine vision module 9130 may use deep learning techniques to bothdetect and recognize/classify objects in the image/vision data. In someembodiments, as shown, the machine vision module 9130 may use separatecircuits and different techniques (e.g., different machine-learnedmodels) to detect and classify objects.

In some embodiments, the machine vision module 9130 may leverage imagedata stored in image library 9152. For example, the machine visionmodule 9130 and/or the processing core(s) 9106 may cause the objectdetection circuit 9134 and/or the object classification circuit 9136 tobe trained to recognize/classify objects based on training data storedin the image library 9152. Examples of image/object classification aredescribed in greater detail throughout the disclosure. In someembodiments, trained models may be stored as intelligence modules 9158.Thus, for example, the chip 9100 may be configured to recognize objectsin a particular environment by storing images of the objects in theimage library 9152 for training purposes, and/or by storing customizedintelligence modules 9158 trained for a particular environment.

In embodiments, the orientation module 9140 may receive various datafrom inputs 9192 and/or data from other modules of the chip 9100 and mayprocess the various data to determine orientation data relating to themobile system. In some embodiments, the environment digital twin circuit9142 may construct and/or update an environment digital twin based oninputs 9192, and/or may retrieve the stored environment digital twin9154. For example, the environment digital twin circuit 9142 may useLIDAR data, radar data, sonar data, and/or the like to determineobjects, surfaces, or other environment features nearby the mobilesystem. In some cases, the environment digital twin circuit 9142 mayupdate the stored environment digital twin 9154 based on data detectedfrom inputs 9192. For example, if the stored environment digital twin9154 indicates that a particular object is at a particular location, butthe environment digital twin circuit 9142 detects that the object isactually at a second location (e.g., based on objects classified by themachine vision system), the environment digital twin 9154 may be updatedwith the correct location information for the object.

The location determination circuit 9144 may use various techniques todetermine a location. For example, the location determination circuitmay compare the environment digital twin generated by the environmentdigital twin circuit 9142 to a pre-stored environment digital twin 9154to determine a position of the mobile system (e.g., if the environmentdigital twin circuit 9142 detects several stationary objects nearby themobile system, and the same objects are located in a particular room ofthe pre-stored environment digital twin 9154, then the locationdetermination circuit 9144 may determine where the mobile system islocated in the particular room). In some embodiments, the locationdetermination circuit 9144 may reconcile location data obtained from thelocation module, relative motion data obtained from the relative motionmodule, object detection and classification data obtained from themachine vision module, the environment digital twin generated by theenvironment digital twin circuit 9142, and/or any pre-stored environmentdigital twin 9154 in order to accurately determine the mobile system'scurrent location within a particular environment. Thus, the locationdetermination circuit 9144 may leverage any of the data inputs 9192and/or data generated by other modules of the chip 9100 to provide anaccurate determination of the location of a mobile system.

In embodiments, the pose determination circuit 9146 may determine poseinformation based on data associated with wheels, limbs, instruments,appendages, or other components of the mobile system. For example, basedon the location and/or relative motion data associated with the variouscomponents, the pose determination circuit 9146 may determine that themobile system is currently sitting, standing, fallen over, movingforward, moving in reverse, and/or the like. The pose determinationcircuit 9146 may compare the location and/or relative motion dataassociated with the various components to data within one or more systemspecifications 9156 to determine the current pose information.Accordingly, the chip 9100 may be configured to work with a particularmobile system by storing a system specification 9156 for that mobilesystem in the storage 9150.

In embodiments, the orientation circuit 9148 may process some or all ofthe various data generated by other circuits and/or modules and/orreceived via input interface 9102 in order to generate orientation datafor transmitting as outputs 9194. For example, the orientation circuit9148 may format the data, place it in various data structures, reconcilethe data, error check the data, and perform other such functions beforetransmission as outputs 9194.

FIG. 110 illustrates a network enhancement chip 9200, one or more ofwhich may be used to enhance the operation and/or performance ofcommunication network(s) as described herein. The chip 9200 may be usedby any value chain entity that leverages communication networks. Inembodiments, the chip(s) 9200 may use artificial intelligence (AI) andother techniques to analyze, predict, optimize, and reconfigure thecommunication network(s). In some of these embodiments, the networkenhancement chip 9200 can leverage (e.g., generate, access, update,process, render, and/or otherwise leverage) a network digital twin toanalyze, predict, optimize, and reconfigure the network. A networkdigital twin can provide a virtual representation of the physicalcommunication network(s) that a network device has access to and thecurrent state of those network(s) and/or network devices, as explainedin more detail below. For example, the network digital twin may indicatea set of available communication networks (e.g., LAN networks, WIFInetworks, cellular networks (e.g., 4G, 5G, and the like), satellitenetworks, Bluetooth networks, RFID networks, and/or the like) to adevice or set of devices, the respective networks to which the device orrespective devices are connected or have connected to in the past,real-time data relating to each respective network (e.g., current dataflows, current bandwidth metrics, current throughput metrics, currenterror rates, current traffic types, etc.), historical data relating toeach respective network (e.g., past data flows, historic bandwidthmetrics, historic throughput metrics, historic error rates, historictraffic types, etc.), and/or the like. In embodiments, a networkenhancement chip 9200 may use such information to optimize a network by,for example, predicting which configurations of the network may optimizea particular network characteristic and then reconfiguring a host deviceand/or other devices on the network accordingly (e.g., switch protocols,switch networks, configure a schedule for transmission of data,configure data priorities, configure compression of certain data,configure reformatting of certain data, up-sampling and/or down-samplingof certain data, configure dropping, buffering, or scheduling of certaindata, and/or the like).

As described herein, the chip(s) 9200 may receive one or more inputs9292 from one or more network(s) and perform one or more AI-assistedfunctions to analyze, predict, optimize, and configure the network(s)based on the inputs 9292. In embodiments, the inputs 9292 may includenetwork signals (e.g., traffic data and/or data from other networkdevices) and/or information about network signals (e.g., signal strengthor other properties of the network signals). The chip(s) 9200 may thendetermine and transmit outputs 9294 comprising instructions foroptimizing or otherwise reconfiguring the network and/or data beingcommunicated thereon. The chip(s) 9200 may be part of a host device thatmay be anywhere within a network (e.g., a server device, client device,router device, etc.) and/or may be a virtual device hosted in a hardwaredevice. In other words, a host device may include any device that isconnected to a communication network.

In embodiments, the network enhancement chip 9200 is configured toanalyze one or more connected communication network(s) to generatenetwork-specific data and to receive network-specific data from othercomponents of the host device, from other network devices, and/or fromother network enhancement chip(s) 9200. The network enhancement chip mayuse (e.g., analyze or otherwise leverage) the network-specific data toupdate information about the communication network (e.g., updating anetwork digital twin) and to predict future conditions of the network.

In embodiments, the network enhancement chip 9200 may analyze networktraffic data at various levels of granularity. For example, the networkenhancement chip may analyze traffic flows and/or individual datamessages (e.g., packets) based on message headers and/or messagepayloads. Additionally or alternatively, the network enhancement chip9200 may receive messages from other network enhancement chip(s) 9200and/or network devices. Such messages may provide device informationthat may be used by the network enhancement chip 9200 to generate and/orupdate a network digital twin.

In embodiments, the network enhancement chip 9200 may analyze physicalattributes of network signals, such as signal strength, packet errorrates, retransmissions, and/or the like to determine network-specificdata (e.g., data indicating a quality/reliability of one or more networklinks), predict future network conditions (e.g., that a wireless devicewill move out of range), and the like. The network enhancement chip 9200may use this information to generate and/or or update a network digitaltwin.

In embodiments, the network enhancement chip 9200 may use one or moreAI-enhanced techniques to determine optimizations for the network basedon the current state of the network, a past state of the network, or afuture predicted state of the network (e.g., as indicated by historicalnetwork data metrics, predicted network demands, a network digital twin,and/or the like), as described in more detail below. Accordingly, thenetwork enhancement chip 9200 may determine optimizations to trafficflows of the network, specific types or configurations of data carriedon the network, messages on the network, and/or devices on the network,and the predicted effects of these optimizations.

The network enhancement chip 9200 may then initiate and/or perform thenetwork optimizations. For example, the network enhancement chip 9200may be configured to reconfigure the network or a segment thereof (e.g.,by performing traffic shaping or otherwise modifying data flows or otherdata received as inputs 9292) and/or to instruct other devices toreconfigure the network or a segment thereof.

The network enhancement chip 9200 may initiate reconfiguration of thenetwork, traffic flows on the network, data transmitted via the network,devices on the network, etc., as described in more detail below. Inembodiments, the network enhancement chip 9200 may instruct one or morenetwork devices to perform one or more reconfiguration functions inorder to cause an optimization to the network. Additionally oralternatively, the network enhancement chip 9200 may reconfigure thenetwork by re-routing the flows (e.g., switching from one network toanother and/or switching a routing path on a network), changing a formatand/or protocol of the flows, or otherwise modifying the flows.

In embodiments, the network enhancement chip 9200 may reconfigure datatransmitted via the network by processing the data in accordance withone or more optimizations. For example, the network enhancement chip9200 may be configured to compress or decompress data, reformat data,resample data, batch data and schedule data transfer of the batcheddata, and/or the like.

In embodiments, the chip(s) 9200 can be modular component(s) that may beintegrated with one or more networks (e.g., as standalone devices)and/or network device(s) in various ways. For example, multiple networkdevices may each include a network enhancement chip 9200, which maycommunicate with each other in order to exchange information, determineoptimizations, and/or configure the network at various points of thenetwork. To facilitate modularity, the chip(s) 9200 may be providedpartially or completely within a housing (not shown) and may receive theinputs 9292 and/or provide the outputs 9294 via electrical connectors,optical connectors, and/or wireless connectors (e.g., antennae,inductive coils, etc.). Additionally or alternatively, the chip(s) 9200may be integrated with other circuits, processors, systems, etc., eitheron one or multiple substrates/chips.

The chip(s) 9200 may be and/or include one or more system-on-chips(SOCs), integrated circuits (ICs), application-specific integratedcircuits (ASICs), and/or the like, for providing the functionalityattributed to chip 9200 and/or any other functionality. For example, thechip 9200 may be provided as part of a SOC that also provides otherfunctions described herein. In general, the components of the chip 9200may comprise one or more general-purpose processing chips that areconfigured using software instructions or other code, and/or maycomprise special-purpose processing chips (e.g., ASICs) customized toperform the functions described herein.

Multiple chip(s) 9200 may be used to perform the functions describedherein. For example, multiple chip(s) 9200 may use serial, parallel,and/or other processing techniques to perform analysis, optimization,and/or configuration functions more quickly, to perform such functionsmore efficiently by offloading more complex computations from one chip9200 to another chip 9200 with a better power source, and/or the like.As another example, one chip 9200 may be used to provide networkenhancement functionality for one part of the network (e.g., aparticular area covered by a wireless network), while another chip 9200may be used to provide network enhancement functionality for a secondpart of the network (e.g., a different area covered by the same wirelessnetwork).

In embodiments, the physical input interface 9202 receives one or moreinputs 9292 to the network enhancement chip 9200 as described herein.The inputs 9292 may be transmitted to the physical input interface 9202via one or more physical network(s) by other network devices, which mayor may not include corresponding network enhancement chip(s) 9200. Thephysical network(s) may include any form of wired or wireless networks.The inputs 9292 may include one or more of network traffic, informationabout the network, information about network devices, instructions foroptimizing or otherwise configuring the network (e.g., as received fromother network enhancement chip(s) 9200), and/or the like. The inputs9292 may also include data that may be stored in storage 9250, such asprotocols for protocol library 9252, a network digital twin 9254 (e.g.,a digital representation of the network), one or more systemspecification(s) 9256, and/or one or more intelligence module(s) 9258.

As stated above, the output data 9294 transmitted from the physicaloutput interface 9204 may include network traffic, information about ahost device that includes the network enhancement chip (e.g., for use byanother network enhancement chip 9200, and/or instructions to optimizeor otherwise configure the network (e.g., to be sent to other networkdevices and/or network enhancement chip(s) 9200). In embodiments, theoutputs of the chip 9200 may be transmitted by the physical outputinterface 9204 via any of the physical network(s) connected to the hostdevice.

In embodiments, the chip 9200 may include one or more of a networkanalysis module 9210, an optimization module 9220, a data configurationmodule 9230, and a network configuration module 9240. In embodiments,the network analysis module 9210 may comprise circuits 9212-9216 foranalyzing the network based on inputs 9292 and/or generating/updating anetwork digital twin. Additionally or alternatively, the chip 9200 mayinclude an optimization module 9220 comprising circuits 9222-9228 forpredicting one or more optimizations to the network based on the inputs9292 and/or a network digital twin. Additionally or alternatively, thechip 9200 may include a data configuration module 9230 comprisingcircuits 9232-9236 for configuring/optimizing network data received asinputs 9292 and transmitting the configured/optimized network data asoutputs 9294. Additionally or alternatively, the chip 9200 may include anetwork configuration module 9240 comprising circuits 9242-9246 forreceiving traffic flows as inputs 9292, configuring/optimizing thetraffic flows, transmitting instructions to other network devices inorder to cause configuration/optimization of the traffic flows, andoutputting the configured/optimized traffic flows and/or instructions asoutputs 9294. The functionalities of the various circuits of the modules9210, 9220, 9230, and/or 9240 are described in more detail below.

The processing core(s) 9206 may comprise one or more processing core(s)that may be configured to perform any of the functions attributed to thechip 9200, either with or without the assistance of the various modules9210, 9220, 9230, and/or 9240. For example, the processing core(s) 9206may leverage and/or invoke various modules to perform various functionsdescribed herein. The processing core(s) 9206 may comprisegeneral-purpose and/or special-purpose processors. In embodiments, theprocessing core(s) 9206 may use serial, parallel, and/or otherprocessing techniques to accomplish the functions described herein.

Accordingly, the processing core(s) 9206 may perform functions inaddition to the functions provided by the various modules 9210, 9220,9230, and/or 9240. For example, the processing core(s) may receive anoutput of one module (e.g., an optimization determined by optimizationmodule 9220) and provide it as input to another module (e.g., to thedata configuration module 9230 and/or network configuration module9240). The processing core(s) 9206 may also process the output of any ofthe module(s) to convert the output into a different format.

In embodiments, the processing core(s) 9206 may further operate to storeand/or retrieve data to/from storage 9250. For example, the processingcore(s) 9206 may store and retrieve protocols in a protocol library 9252(e.g., for use by the various modules, as described in more detailbelow), may store and retrieve a network digital twin 9254 (e.g., asgenerated/updated or otherwise leveraged by the various modules, asdescribed in more detail below), may store and retrieve systemspecification(s) 9256 (e.g., for determining information about variousnetwork devices), and/or may store and retrieve intelligence module(s)9258 for implementing the various functions described herein. Inembodiments, the processing core(s) may implement any of thefunctionalities of the intelligence service 8800 (as described withrespect to FIG. 104 ) using the intelligence modules 9258 (which mayinclude one or more of the artificial intelligence modules 8804 of FIG.104 ).

The network analysis module 9210 may receive network signals (e.g.,network traffic between various network endpoint devices, messagesincluding information about network devices, etc.), information aboutnetwork signals (e.g., signal strength or other physical attributes ofnetwork signals), and/or other network information (e.g., dataindicating current or historical network performance, current orhistorical network device information, network digital twin(s) generatedby other devices, etc.) and determine information about the network, aswell as generate and/or update one or more network digital twin(s)corresponding to various communication network(s).

In embodiments, the signal analysis circuit 9212 may receive networksignals from the inputs 9292 and perform signal analysis (e.g., analysisof header information and/or payload information) to determineinformation about the signal. For example, the signal analysis circuit9212 may analyze whether network traffic belongs to a certain trafficflow based on header information (e.g., from/to addresses, protocols,flow identifiers, etc.) and/or payload information (e.g., based on thetype of data included in the payload, whether the data is encrypted,etc.). As another example, the signal analysis circuit 9212 may detectmessages that include device information about a network device.Additionally or alternatively, the signal analysis circuit 9212 mayanalyze physical attributes of the signals received as inputs 9292, suchas signal strength indicators. In these embodiments, the signal analysiscircuit 9212 may further analyze the physical attributes over time(e.g., to determine that a signal strength has been weakening and/orpredict that a corresponding wireless link is likely to be lost). Thesignal analysis circuit 9212 may analyze all or only some of any networktraffic received as inputs 9292. For example, the signal analysiscircuit may sample one of every N network packets received as inputs9292, analyze the physical attributes of signals every N number ofmicroseconds, and/or the like).

In embodiments, the data analysis circuit 9214 may determine furthernetwork information based on data within the analyzed signals. Forexample, the data analysis circuit 9214 may analyze whether the data fora particular traffic flow is encrypted, compressed, has a particularformat, is associated with a particular priority level (e.g., a prioritylevel associated with a contracted data rate), or the like. The dataanalysis circuit 9214 may add such information to one or morecorresponding network digital twins, each of which may be specific to aparticular communication network carrying the data, one or more deviceson the network, one or more data configurations for the network, one ormore rate schedules for the network, etc. In embodiments, the dataanalysis circuit 9214 may analyze application-specific data that mayindicate a particular application and/or one or more attributes (e.g.,whether the data is payment data, customer data, whether the data isassociated with a particular project, etc.). In these embodiments, thedata analysis circuit 9214 may add such information to a network digitaltwin. Additionally or alternatively, the data analysis circuit 9214 mayanalyze received messages to detect information about network devices.For example, the data analysis circuit 9214 may analyze data (e.g., aMAC address or other identifier) included in a data message to identifya particular manufacturer, model, or identity of a network device. Inthese cases, the data analysis circuit 9214 may then retrieve additionalinformation about the identified network device using a systemspecification 9256 corresponding to the identified device. Additionallyor alternatively, the data analysis circuit 9214 may analyze statusmessages that indicate a current state of a network device, such as abattery level, current available bandwidth, current available processingcapability, and/or the like. The data analysis circuit 9214 may storeinformation about various network devices in a network digital twin 9254corresponding to a particular network.

In embodiments, the network diagnostic circuit 9216 may then determinenetwork information based on the analyzed signals and/or data. Forexample, the network diagnostic circuit 9216 may detect and record aprotocol, format, endpoint devices, bandwidth and/or throughput (e.g.,current, average, minimum, and/or maximum bandwidth/throughput), errorrate, packet loss rate, flow priority, flow quality of service (QoS)metrics/requirements, flow schedule, applications-specific data, and/orthe like, for each traffic flow on one or more connected networks. Asanother example, the network diagnostic circuit 9216 may detect a newtraffic flow and add it to a list of traffic flows for a particularnetwork. The network diagnostic circuit 9216 may also determinediagnostic information indicating errors or other conditions of thenetwork. For example, if the network diagnostic circuit 9216 detectsthat no traffic is being received via a particular network or from aparticular device, it may detect that the corresponding network/deviceis unavailable. In embodiments, the network diagnostic circuit 9216 mayperform diagnostic workflows in order to detect problems or otherconditions on the network. For example, the network diagnostic circuit9216 may poll network devices for status information, attempt totransmit data through one or more communication networks, send orreceive test data flows to measure bandwidth, throughput, etc., andperform other such diagnostic functions. In embodiments, the networkdiagnostic circuit 9216 may use the determined network/diagnosticinformation to generate or update one or more network digital twins 9254corresponding to a particular network, network device, dataconfiguration, rate schedule, and/or the like.

In embodiments, the optimization module 9220 may leverage the networkanalysis outputted by the network analysis modules 9210 and/or networkdigital twin(s) 9254 to determine one or more optimizations for thenetwork and the predicted effects of the optimizations. The optimizationmodule 9220 may use AI-assisted functions (e.g., machine-learned modelsor other intelligence modules 9258) to predict that certainoptimizations will improve the functioning of the network, schedule ofthe network, quality of data transmitted via the network, security ofdata transmitted via the network, and the like.

In embodiments, the data optimization circuit 9222 may predict theeffects one or more optimizations to be applied to network data. Forexample, the data optimization circuit 9222 may leverage intelligencemodules 9258 (e.g., trained deep learning models) and/or storedoptimization parameters to determine, based on current networkinformation, that a particular type of data should be re-routed (e.g.,through a different network), compressed, down-sampled, dropped,buffered, and/or re-scheduled in order to optimize a particular networkmetric. The optimization parameters may be specified by one or moresystem specifications 9256, and thus the data optimization circuit 9222may be configured to optimize communications networks in specified waysby storing corresponding system specifications in storage 9250. Asanother example, the data optimization circuit 9222 may use AI-assistedtechniques (e.g., leveraging intelligence modules 9258) to determinethat the network has sufficient capacity to increase the quality of datatransmitted via the network, such as by up-sampling, uncompressing,providing a higher priority to, or otherwise increasing the quality ofdata transmitted via the network. In this example, a systemspecification 9256 may indicate that the data optimization circuit 9222should optimize for increased data quality in general, increased dataquality for a particular application, flow, type of data,sending/receiving network device, and/or the like. Accordingly, the dataoptimization circuit 9222 may leverage AI techniques to optimize variousnetwork parameters as required by a particular system specification.

In embodiments, the network optimization circuit 9224 may determine oneor more optimizations to be applied to network devices. For example, thenetwork optimization circuit 9224 may leverage intelligence modules 9258(e.g., trained deep learning models) to determine, based on currentnetwork information, that a particular network device should performparticular actions (e.g., power up or down, switch networks, adjust atransmission schedule of another device, adjust a protocol used byanother network device, re-route traffic from another device, performcompression or some other data modification on all traffic sent orreceived by another device, and/or the like) to improve networkperformance or optimize for some other parameter (e.g., as indicated bya system specification 9256). Similarly, the network optimizationcircuit 9224 may determine that network devices should take certainactions to improve the quality of data transmitted via the networkand/or perform any other optimizations.

In embodiments, the data optimization circuit 9222 and/or the networkoptimization circuit 9224 may leverage the network security circuit 9226and/or the network governance circuit 9228 as part of determiningoptimizations to data and/or the network. The network security circuit9226 may enforce security rule(s) that may alter and/or may overrideoptimizations proposed by the data optimization circuit 9222 and/or thenetwork optimization circuit 9224. For example, the network securitycircuit 9226 may analyze proposed optimizations to the networkdevice(s), data, and/or network(s) to determine whether the proposedoptimizations are sufficiently secure or insufficiently secure, orotherwise comply with security rules. As a specific example, the networksecurity circuit 9226 may determine that a proposed optimizationinvolving decryption of network traffic may be insecure for a particulardata type or traffic flow, and thus may override and/or alter theproposed optimization.

In a similar manner, the network governance circuit 9228 may enforcegovernance rules that specify certain legal requirements, businessrequirements, technical requirements, and the like. Accordingly, thenetwork governance circuit 9228 may alter and/or may overrideoptimizations proposed by the data optimization circuit 9222 and/ornetwork optimization circuit 9224 so that the optimizations will complywith the governance rule(s). In embodiments, the network securitycircuit 9226 and/or network governance circuit 9228 may leverageintelligence modules 9258 that store and/or otherwise specify thesecurity and/or governance rules. In embodiments, the network securitycircuit 9226 and/or network governance circuit 9228 may implement any ofthe functionalities of the intelligence service 8800 (as described withrespect to FIG. 104 ) using the intelligence modules 9258 (which mayinclude one or more of the artificial intelligence modules 8804 of FIG.104 ).

In embodiments, the data configuration module 9230 may receive datatraffic via inputs 9292 and apply any optimizations determined by thedata optimization circuit 9222 and/or by the network optimizationcircuit 9224 to the received data traffic before transmitting theoptimized data traffic as outputs 9294. The data capture/extractioncircuit may receive inbound or outbound data packets (e.g., from othernetwork devices and/or from the host device) and may extract the datafrom the data packets.

The data encryption circuit 9234 may perform any necessaryencryption/decryption operations on the extracted data. The dataencryption circuit 9234 may decrypt data received from another device sothat the data may be analyzed and data-specific optimizations may beapplied. For example, if the optimization module 9220 indicates thatdata should be re-formatted (e.g., up- or down-sampled,compressed/decompressed, and/or the like), the data may first need to bedecrypted by the data encryption circuit 9234 before the optimizationsmay be applied. Additionally or alternatively, the data encryptioncircuit 9234 may apply encryption to the data if the optimization module9220 determines that data encryption should be applied (e.g., toincrease the security of a particular type of data or traffic flow).

The data processing circuit 9236 may perform any processing on the datato implement the optimizations determined by the optimization module9220. For example, if the data optimization circuit 9222 and/or thenetwork optimization circuit 9224 determine that data associated withcertain attribute(s) (e.g., a particular type of data, particular dataflow, particular application-specific attribute, particular datapriority, particular data protocol, etc.) should be optimized by beingprocessed in a certain way (e.g., by compressing/decompressing,up-sampling or down-sampling, reformatting, delaying, buffering,rescheduling, etc.), then the data processing circuit may perform theprocessing when it detects data that matches the attribute(s). Thus, thedata processing circuit 9236 may perform data optimizations on data thatis received by the network enhancement chip 9200.

In embodiments, the network configuration module 9240 may transmit andreceive signals to/from the communication network in order to performcertain optimizations to the network and/or network devices asdetermined by the network optimization circuit 9224. The networkconfiguration module 9240 may perform network optimization in parallelto or sequentially before or after the data configuration module 9230optimizes data.

In embodiments, the signal processing circuit 9242 may generate andreceive inbound or outbound data signals (e.g., to/from other networkdevices and/or to/from the host device comprising the networkenhancement chip 9200) to coordinate with other network enhancementchip(s) 9200 and/or network devices on the network. For example, anetwork enhancement chip 9200 may transmit a signal to a target networkdevice that instructs the target network device to perform some action(e.g., as determined by the network optimization circuit 9224) tooptimize the network. Additionally or alternatively, the signalprocessing circuit 9242 may receive instructions from other networkenhancement chip(s) on the network that instruct the network enhancementchip 9200 to perform configuration functions in order to optimize thenetwork.

In embodiments, the signal processing circuit 9242 may modify signalsbeing sent to other network devices based on optimizations determined bythe optimization module 9220. For example, if the signal processingcircuit 9242 detects a message being sent (e.g., by another networkdevice) that will cause a target network device to use a first protocol,but the optimization module 9220 determined that using a second protocolwill optimize the network, then the signal processing circuit 9242 maymodify the message to instead instruct the use of the second protocol.Similarly, a signal processing circuit 9242 may drop (e.g., deletewithout transmission) or delay a message being sent to another device ifthe message contains an instruction that conflicts with an optimizationdetermined by the optimization module 9220. Thus, the signal processingcircuit 9242 may cause optimizations by delaying or overriding variousinstructions sent and received by various network devices.

In embodiments, the protocol switching circuit 9244 may configure aprotocol of data signals being transmitted across the network. As aspecific example, the protocol switching circuit 9244 may switch acertain type of data or data flow from a TCP/IP protocol to a UDP/IPprotocol in order to optimize a particular network parameter. Theprotocol switching circuit 9244 may access protocol information from aprotocol library 9252 in order to configure one or more protocols. Inembodiments, the protocol switching circuit 9244 may reconfigure otherprotocol-level attributes of signals and/or other data to be transmittedacross the network. For example, the protocol switching circuit 9244 mayreconfigure a source or destination address, a protocol time stamp, aprotocol stream identifier, and/or any other fields of a protocolheader. Additionally or alternatively, the protocol switching circuit9244 may generate instructions for transmission to another networkdevice that may cause the other network device to reconfigure theprotocol of data signals being transmitted by that network device. Theprotocol switching circuit 9244 may reconfigure the protocols of trafficon the network based on optimizations determined by the optimizationmodule 9220. Additionally or alternatively, the protocol switchingcircuit 9244 may reconfigure the protocols based on a current state ofthe network (e.g., as indicated by the network digital twin 9254) and/orbased on the processing performed by the signal processing circuit 9242.

In embodiments, the network switching circuit 9246 may reconfigure therouting, scheduling, network topology, or other attributes of one ormore network(s) that are in communication with the network enhancementchip 9200. For example, the network switching circuit 9246 mayreconfigure a network from a mesh topology to a star topology (e.g., byinstructing one or more network devices to change roles), route trafficacross one network instead of another network (e.g., to balanceavailable bandwidth on the two networks), route traffic through onerouter instead of another router (e.g., to balance load on the tworouters), schedule transmission of first traffic in a first transmissionslot and second traffic in a second transmission slot, and/or the like.In some cases, the network switching circuit 9246 may reconfigure therouting and/or scheduling of data received by the network enhancementchip 9200 (e.g., as inputs 9292). Additionally or alternatively, theprotocol switching circuit 9244 may generate instructions fortransmission to another network device that may cause the other networkdevice to reconfigure an aspect of the network. The network switchingcircuit 9246 may reconfigure the network based on optimizationsdetermined by the optimization module 9220. Additionally oralternatively, the network switching circuit 9246 may reconfigure thenetwork based on a current state of the network (e.g., as indicated bythe network digital twin 9254) and/or based on predictions/analysisgenerated by the signal processing circuit 9242.

FIG. 111 illustrates a diagnostic chip 9300, one or more of which may beused to perform one or more diagnostic functions as described herein.The chip 9300 may be used by any value chain entity that performsdiagnostics. In embodiments, the chip(s) 9300 may use artificialintelligence (AI) and other techniques to perform diagnostics based ondata from one or more sensors, including biological sensors, chemicalsensors, and/or electromechanical sensors, and to generate reportsincluding analyses and recommended actions based on the diagnostics. Inembodiments, the diagnostic chip 9300 may be configured to perform oneor more particular diagnostics by receiving, storing, and leveragingcorresponding specifications that indicate the types of sensor inputs,how to process and format the sensor inputs, how to analyze the sensorinputs, etc. Similarly, the diagnostic chip 9300 may be configured toperform certain diagnostics by receiving, storing, and leveragingcorresponding analytics libraries and/or intelligence modules that maybe used to configure and perform one or more analyses.

In embodiments, the chip 9300 may be configured or reconfigured toreceive and interpret data from a wide variety of sensors, including,without limitation, chemical sensors (e.g., hazard-specific sensors,flammability sensors, compound-specific sensors, etc.), biologicalsensors (e.g., bio-hazard materials and/or hazard levels sensors,radiation sensors, etc.), electro-mechanical sensors (e.g., vibrationsensors, stress/strain sensors, electrical resistance/current sensors,sensors that measure motion and/or location data such as inertia, speed,acceleration, GPS, etc.), optical/imaging (e.g., light sensors,hyperspectral sensors, intensity sensors, thermal sensors, etc.) andother environmental sensors (e.g., temperature sensors, humiditysensors, air movement sensors, etc.), and the like. The chip 9300 may bereconfigured to receive and interpret specific sensor data based onsensor specification(s) that enable the chip 9300 to receive andinterpret sensor data from the corresponding sensors.

In embodiments, the chip 9300 may be configured or reconfigured toperform organic analyses, lab analyses, and/or electromechanicalanalyses based on the sensor data. For example, the chip 9300 mayinclude lab-on-chip and/or organ-on-chip functionality that may allow itto simulate organisms, perform lab analyses, perform electromechanicalanalyses, etc. The chip 9300 may receive, store, and leverage specificanalytics libraries and/or intelligence modules that enable the chip9300 to perform corresponding simulations/analyses, make predictionsusing corresponding AI techniques (e.g., using deep learning modelstrained to interpret corresponding sensor data), and the like. Usingsimilar techniques, the chip 9300 may further combine the results ofvarious analyses in order to perform one or more combined analyses.

In embodiments, the chip 9300 may be configured to use governancelibraries to control analyses, make predictions, and/or providerecommendations. For example, governance libraries may indicate whetherparticular conditions are acceptable or not, and thus may controlwhether actions should be taken to address a condition. The chip 9300may be configured to report the results of any analyses, includingcurrent or predicted conditions, recommended actions to address theconditions, and the like.

In embodiments, the chip(s) 9300 can be modular component(s) that may beintegrated with a host system in various ways. For example, the chip(s)may be integrated with a mobile host system (e.g., a robot), astationary host system, or any other host system that receives sensorinputs. To facilitate this modularity, the chip(s) 9300 may be providedpartially or completely within a housing (not shown) and may receive theinputs 9392 and/or provide the outputs 9394 via electrical connectors,optical connectors, and/or wireless connectors (e.g., antennae,inductive coils, etc.). Additionally or alternatively, the chip(s) 9300may be integrated with other circuits, processors, systems, etc., eitheron one or multiple substrates/chips.

The chip(s) 9300 may be and/or include one or more system-on-chips(SOCs), integrated circuits (ICs), application-specific integratedcircuits (ASICs), and/or the like, for providing the functionalityattributed to chip 9300 and/or any other functionality. For example, thechip 9300 may be provided as part of a SOC that also provides otherfunctions described herein. In general, the components of the chip 9300may comprise one or more general-purpose processing chips that areconfigured using software instructions or other code, and/or maycomprise special-purpose processing chips (e.g., ASICs) customized toperform the functions described herein.

Multiple chip(s) 9300 may be used to perform the functions describedherein. For example, multiple chip(s) 9300 may use serial, parallel,and/or other processing techniques to perform analyses more quickly, toperform analyses more efficiently by offloading more complexcomputations from one chip 9300 to another chip 9300 with a better powersource, and/or the like. As another example, one chip 9300 may be usedto provide a first analysis and a second analysis, while another chip9300 may be used to provide a combined analysis based on the firstanalysis and the second analysis.

In embodiments, the physical input interface 9302 receives one or moreinputs 9392 to the diagnostic chip 9300 as described herein. The inputs9392 may be transmitted to the physical input interface 9302 by otherchips, circuits, modules, and/or other components of the host system, orby other devices in communication with the host system (e.g., via acommunication network). For example, the input data may come fromsensors, sensor-processing chips/modules/circuits, antennae, storagedevices, network interfaces, or any other source of data for the chip(s)9300 as described herein. The physical input interface 9302 may connectwith the source(s) of the inputs 9392 via wired or wireless connections.As state above, the inputs 9392 may include any type of sensor data. Theinputs 9392 may also include data that may be stored in storage 9350,such as analytics rules/configurations for analytics library 9352,governance rules/configurations for a governance library 9354, one ormore system specification(s) 9356 (e.g., sensor specifications), and/orone or more intelligence module(s) 9358.

The output data 9394 transmitted from the physical output interface 9304may include report(s) indicating the results of the analyses, particularconditions indicated by the analyses, predictions, other diagnosticsinformation, and/or recommended actions to address any particularconditions or predicted conditions. In embodiments, the outputs of thechip 9300 may be transmitted by the physical output interface 9304 toother chips, circuits, modules, and/or other components of a host systemor another device in communication with the host system as describedherein. The physical output interface 9304 may connect to thesecomponents via wired or wireless connections.

In embodiments, the chip 9300 may include one or more of a sensor module9310, an analysis module 9320, and/or an output module 9330. Inembodiments, the sensor module 9310 may comprise circuits 9312-9318 forreceiving and performing initial processing (e.g., filtering) on sensordata received as inputs 9392. Additionally or alternatively, the chip9300 may include an analysis module 9320 comprising circuits 9322-9326for performing analyses, detecting conditions, predicting futureconditions, generating other diagnostic information, and generatingrecommendations for addressing any conditions. Additionally oralternatively, the chip 9300 may include an output module 9330comprising circuits 9332-9336 for performing additional combinedanalyses, enforcing governance rules on the analyses, predictions,recommendations, etc., and outputting a report includingdiagnostic/analysis data. The functionalities of the various circuits ofthe modules 9310, 9320, and/or 9330 are described in more detail below.

The processing core(s) 9306 may comprise one or more processing core(s)that may be configured to perform any of the functions attributed to thechip 9300, either with or without the assistance of the various modules9310, 9320, and/or 9330. For example, the processing core(s) 9306 mayleverage and/or invoke various modules to perform various functionsdescribed herein. The processing core(s) 9306 may comprisegeneral-purpose and/or special-purpose processors. In embodiments, theprocessing core(s) 9306 may use serial, parallel, and/or otherprocessing techniques to accomplish the functions described herein.

Accordingly, the processing core(s) 9306 may perform functions inaddition to the functions provided by the various modules 9310, 9320,and/or 9330. For example, the processing core(s) may receive an outputof one module (e.g., sensor data output by the sensor module 9310) andprovide it as input to another module (e.g., to the analysis module9320). The processing core(s) 9306 may also process the output of any ofthe module(s) to convert the output into a different format.

In embodiments, the processing core(s) 9306 may further operate to storeand/or retrieve data to/from storage 9350. For example, the processingcore(s) 9306 may store and retrieve analytics configurations/data in ananalytics library 9352 and/or governance configurations/data in agovernance library 9354 (e.g., for use by the analysis module 9320, asdescribed in more detail below), may store and retrieve systemspecifications 9356 (e.g., for configuring the sensor module 9310, asdescribed in more detail below), and/or may store and retrieveintelligence module(s) 9358 for implementing the various functionsdescribed herein. In embodiments, the processing core(s) may implementany of the functionalities of the intelligence service 8800 (asdescribed with respect to FIG. 104 ) using the intelligence modules 9358(which may include one or more of the artificial intelligence modules8804 of FIG. 104 ).

The sensor module 9310 may receive and perform initial processing onsensor data from any type of sensor. In some embodiments, the biologicalsensing circuit 9312 may receive and/or process (e.g., filter, sanitycheck, error check, etc.) sensor data from biological sensors.Additionally or alternatively, the chemical sensing circuit 9314 mayreceive and/or process (e.g., filter, sanity check, error check, etc.)sensor data from chemical sensors. Additionally or alternatively, theelectromechanical sensing circuit 9316 may receive and/or process (e.g.,filter, sanity check, error check, etc.) sensor data from electricalsensors, mechanical sensors, and/or electromechanical sensors.Additionally or alternatively, the environmental sensing circuit 9318may receive and/or process (e.g., filter, sanity check, error check,etc.) sensor data from environmental sensors, including atmosphericsensors, imaging sensors, and/or the like.

In embodiments, each of the biological sensing circuit 9312, thechemical sensing circuit 9314, and/or the electromechanical sensingcircuit 9316 may access system specifications 9356 corresponding toparticular sensors in order to configure the sensing circuit to processcorresponding sensor data. For example, when the diagnostic chip 9300 isconfigured to perform a particular organic analysis (e.g.,prediction/simulation/testing of a particular organ or organ system),the biological sensing circuit 9312 may retrieve the systemspecifications 9356 for corresponding sensors (e.g., microfluidicsensors, bioMEMS sensors, etc.) so that the biological sensing circuit9312 may receive and process (e.g., format, filter, error check, etc.)the relevant sensor data. As another example, when the diagnostic chip9300 is configured to perform a particular lab analysis (e.g., drugtesting, disease testing etc.), the chemical sensing circuit 9314 mayretrieve the system specifications 9356 for corresponding sensors (e.g.,chemical sensors) so that the chemical sensing circuit 9314 may receiveand process (e.g., format, filter, error check, etc.) the relevantsensor data. As another example, when the diagnostic chip 9300 isconfigured to perform electromechanical analysis (e.g., a diagnosticanalysis of a particular machine/circuit based on vibration sensors,electric sensors, electromechanical sensors, etc.), theelectromechanical sensing circuit 9316 may retrieve the systemspecifications 9356 for corresponding sensors (e.g., MEMS sensors,vibration sensors, etc.) so that the electromechanical sensing circuit9316 may receive and process (e.g., format, filter, error check, etc.)the relevant sensor data. As another example, when the diagnostic chip9300 is configured to perform environmental analysis (e.g., a diagnosticanalysis based on imaging data and/or environmental data), theenvironmental sensing circuit 9318 may retrieve the systemspecifications 9356 for corresponding sensors (e.g., imaging sensors,optical sensors, other environmental sensors, etc.) so that theenvironmental sensing circuit 9318 may receive and process (e.g.,format, filter, error check, etc.) the relevant sensor data.

The analysis module 9320 may receive processed sensor data from thesensor module 9310 and perform various analyses using the organicanalysis circuit 9322, lab analysis circuit 9324, and/orelectromechanical analysis circuit 9326. Each of the organic analysiscircuit 9322, lab analysis circuit 9324, electromechanical analysiscircuit 9326, and/or environmental analysis circuit 9328 may retrieveanalytics configuration(s) from analytics library 9352 and/orintelligence module(s) 9358 in order to perform relevant analyses. Forexample, when the diagnostic chip 9300 is configured to performsimulation of a particular organ or organ system, the organic analysiscircuit 9322 may retrieve analytics data from analytics library 9352specifying configuration parameters corresponding to the organ/organsystem (e.g., a particular biology, functional mechanisms, etc.) and mayretrieve an intelligence module 9358 trained to predict and/or analyze aresponse of the organ/organ system to physiological stimuli, particulardrugs, particular diseases, and/or other inputs. Similarly, when thediagnostic chip 9300 is configured to perform disease testing/analysis,the lab analysis circuit 9324 may retrieve analytics data from analyticslibrary 9352 specifying configuration parameters corresponding to thedisease (e.g., particular indicators, symptoms, etc.) and may retrievean intelligence module 9358 trained to predict a progression of thedisease, a response of the disease to treatment, and/or the like.Similarly, when the diagnostic chip 9300 is configured to performdiagnostic analysis of a machine, the electromechanical analysis circuit9326 may retrieve analytics data from analytics library 9352 specifyingconfiguration parameters for the machine (e.g., frequencies and/orfrequency patterns indicating particular states of the machine orsub-parts of the machine, electrical information indicating correct orincorrect operating levels for electrical circuits of the machine, etc.)and may retrieve an intelligence module 9358 trained to predict apotential breakdown or other condition of the machine, effects ofmaintenance actions, etc. Similarly, when the diagnostic chip 9300 isconfigured to perform a diagnostic environmental analysis, theenvironmental analysis circuit 9328 may retrieve analytics data fromanalytics library 9352 specifying configuration parameters for theenvironment (e.g., image/optical data and/or other environmental dataindicating particular conditions of the environment, etc.) and mayretrieve an intelligence module 9358 trained to predict a potentialenvironmental condition such as conditions that are safe/unsafe forhumans and/or other environmental conditions.

Each of the organic analysis circuit 9322, lab analysis circuit 9324,electromechanical analysis circuit 9326, and/or environmental analysiscircuit 9328 may use one or more AI-assisted techniques to performanalyses, determine/predict conditions, predict the effects oftreatments/maintenance/preventative actions, and/or the like. Forexample, one of the circuits may configure (e.g., using configurationparameters specified by an analytics library) a first AI-assistedtechnique to detect a particular condition (e.g., a gradient-boostedtrees model), and then the same or another circuit may use a differentAI-assisted technique (e.g., a neural network trained using deeplearning techniques) to predict the response to a treatment plan for theparticular condition. Similarly, the chip 9300 may use multipleAI-assisted techniques in order to perform the same tasks in order toimprove the accuracy of diagnostic information. Thus, by leveragingmultiple AI-assisted techniques, the chip 9300 may be capable ofperforming complex and highly accurate workflows that leverage differentAI-assisted techniques.

In embodiments, multiple intelligence module(s) 9358 may be used toprovide different types of diagnostics for a single workflow. Inembodiments, the intelligence modules 9358 may include one or more ofthe artificial intelligence modules 8804 of FIG. 104 . Additionally oralternatively, multiple of the analysis circuits 9322-9328 may be usedfor an analysis workflow. For example, an analysis for diseasediagnostics applications may use both chemical and biological sensors asinputs, and the chip 9300 may correspondingly use both the organicanalysis circuit 9322 and/or the lab analysis circuit 9324 to performaspects of the relevant analysis.

In embodiments, the output module 9330 may perform combined analysesusing the outcomes of the analysis module 9320, may enforce governancerules, and/or may generate/transmit reports including the results of theanalyses generated by the analysis module 9320 and/or the combinedanalysis circuit 9332.

The combined analysis circuit 9332 may correlate and further analyzemultiple analyses generated by the analysis module 9320. For example, ifa first diagnostic analysis (e.g., using a first AI-assisted techniqueand/or a first set of sensor inputs) indicated the presence of aparticular condition (e.g., that a disease is present), and a seconddiagnostic analysis (e.g., using a second AI-assisted technique and/or asecond set of sensor inputs) indicated the absence of the particularcondition (e.g., the disease is absent), the combined analysis circuit9332 may combine the results of the first and second diagnosticanalysis, apply weightings, leverage intelligence modules 9358, and/orotherwise process the outputs of the first and second diagnosticanalysis to generate an indication of whether the particular conditionis present, a likelihood of the particular condition being present,and/or the like. In embodiments, the combined analysis circuit 9332 mayprocess a first diagnostic analysis indicating a first condition and asecond diagnostic analysis indicating a second condition to determinethat a third condition is present. In embodiments, the combined analysiscircuit 9332 may combine a first action plan (e.g., atreatment/maintenance/preventative action plan) indicated by a firstdiagnostic analysis and a second action plan indicated by a seconddiagnostic analysis to yield a combined action plan that may includeactions indicated by the first action plan, actions indicated by thesecond action plan, and/or third actions not indicating by either thefirst or second action plan. In embodiments, the combined analysiscircuit 9332 may process a first diagnostic analysis indicating a firstprobability of a condition and a second diagnostic analysis indicating asecond probability of the condition to yield a combined analysisindicating a third probability of the condition, where the thirdprobability may be lower, higher, in between, or equal to one or both ofthe first and second probabilities.

The governance circuit 9334 may enforce rules, override actions inaction plans, control analyses performed by the analysis circuits9322-9328, or otherwise modify the analyses and/or outputs of analysesto conform with governance rules. For example, the governance circuitmay require that certain actions of an action plan are not dangerous tohumans, are not illegal, etc. The governance circuit 9334 may retrievegovernance rules from governance library 9354, which may store rulesthat are tailored for a particular application. For example, when thechip 9300 is monitoring environmental conditions in a location wherehumans work, the governance circuit 9334 may retrieve a governancelibrary specifying acceptable environmental conditions for humans. Thegovernance circuit 9334 may then use this information to require certainactions when certain conditions are detected (e.g., sounding an alarmwhen a dangerous substance is detected), override certain actions in anaction plan (e.g., actions that may change the environment to bedangerous or otherwise unsuitable for humans), control which types ofanalysis are used and/or how the analyses are performed by the variousanalysis circuits, and/or the like. By contrast, when the chip 9300 isconfigured to monitor environmental conditions in a location where thereare no humans, it may use a different set of governance rules. In somecases, governance rules may require reporting of certain conditions tocertain parties (e.g., reporting of disease data to a patient, doctor,etc.), prohibit reporting of conditions to certain parties (e.g., tocomply with HIPAA laws), and/or the like. To control the operations ofthe analysis circuits, the governance circuit 9334 may be configured tomonitor and/or be leveraged by the analysis circuits 9322-9328 such thatthe governance circuit 9334 may instruct the analysis circuits toperform or not perform certain analyses, modify how the analyses areperformed, and/or the like.

The reporting circuit 9336 may generate reports including the results ofthe analyses and/or combined analyses, as modified by any governancerules, and output the reports (e.g., as outputs 9394). The reportingcircuit 9336 may format the data as required to interoperate with anymodule/device/system that receives the outputs 9394. In embodiments, thereporting circuit 9336 may generate human-readable reports including theresults of the analyses and transmit the human-readable analyses to oneor more client devices (e.g., as indicated by system specifications 9356or other configuration parameters).

FIG. 112 illustrates a governance chip 9400, one or more of which may beused to perform one or more governance functions as described herein.The chip 9400 may be used by any value chain entity that conforms withvarious governance standards, including safety, security, quality,regulatory, financial, or other standards. In embodiments, a chip 9400may use artificial intelligence (AI) and other techniques to performgovernance functions on input data from one or more components of a hostdevice incorporating the governance chip 9400 and/or other devices incommunication with the host device. In embodiments, the governance chip9400 may be configured to receive and analyze data to determinesituations in which governance may apply, may be configured to build oneor more models for enforcing governance, and then may enforce rules,limitations, requirements, quality, or other aspects of governance usingthe models by triggering actions in response to governance violations,reconfiguring data to avoid governance violations, issuing instructionsto one or more devices in communication with the governance chip 9400,and/or otherwise performing governance actions using the governance chip9400.

In embodiments, the chip 9400 may be configured to receive input datacomprising a set of data to which governance standards may be applied.The input data may include a data set that must comply with one or moresafety, security, quality, regulatory, financial, or other standards fora particular domain. In embodiments, multiple governance standards mayapply to a single data set. For example, both safety and qualitystandards may apply to a given set of data. The governance standards mayonly apply to the set of data based on certain conditions, such as alocation or other condition of a particular device in communication withthe chip 9400, a current state of a module, device, system, or network,or other such conditions.

Accordingly, the chip 9400 may initially analyze a particular data set(e.g., a data set received as inputs 9492) to determine whether one ormore governance standards apply, as described in more detail below.Based on determining that one or more governance standards apply, thechip 9400 may then prioritize the applicable standards and generateand/or validate a model that enforces the governance standards. Themodel may include one or more flows for checking that data complies withthe governance standards, performing actions to cause compliance withthe governance standards, taking remedial actions when governanceviolations occur, and the like. When multiple governance standardsapply, the chip 9400 may generate a model that reconciles any potentialoverlaps or conflicts between the multiple standards. The chip 9400 mayvalidate a model using test data or other strategies, as described inmore detail below.

After the model is generated and/or validated, the chip 9400 may use themodel to enforce governance standards. The chip 9400 may use the modelto enforce governance standards on one or more received data sets,including data sets that are not received until after the model has beengenerated and validated. In embodiments, the chip 9400 may continuallyoptimize the model over time to ensure governance compliance asconditions change, and may generate reports and other outputs forallowing review of governance enforcement and/or for causing otherdevices to perform enforcement of governance.

In embodiments, the chip(s) 9400 can be modular component(s) that may beintegrated with a host system in various ways. For example, the chip(s)may be integrated with a mobile host system, a stationary host system,or any other host system that receives input data subject to governance.To facilitate this modularity, the chip(s) 9400 may be providedpartially or completely within a housing (not shown) and may receive theinputs 9492 and/or provide the outputs 9494 via electrical connectors,optical connectors, and/or wireless connectors (e.g., antennae,inductive coils, etc.). Additionally or alternatively, the chip(s) 9400may be integrated with other circuits, processors, systems, etc., eitheron one or multiple substrates/chips.

The chip(s) 9400 may be and/or include one or more system-on-chips(SOCs), integrated circuits (ICs), application-specific integratedcircuits (ASICs), and/or the like, for providing the functionalityattributed to chip 9400 and/or any other functionality. For example, thechip 9400 may be provided as part of a SOC that also provides otherfunctions described herein. In general, the components of the chip 9400may comprise one or more general-purpose processing chips that areconfigured using software instructions or other code, and/or maycomprise special-purpose processing chips (e.g., ASICs) customized toperform the functions described herein.

Multiple chip(s) 9400 may be used to perform the functions describedherein. For example, multiple chip(s) 9400 may use serial, parallel,and/or other processing techniques to perform analysis and/or governancefunctions more quickly, to perform analysis and/or governance functionsmore efficiently by offloading more complex computations from one chip9400 to another chip 9400 with a better power source, and/or the like.As another example, one chip 9400 may be used to provide a firstanalysis and governance function, while another chip 9400 may be used toprovide a second analysis and governance function on the same data set.

In embodiments, the physical input interface 9402 receives one or moreinputs 9492 to the governance chip 9400 as described herein. The inputs9492 may be transmitted to the physical input interface 9402 by otherchips, circuits, modules, and/or other components of the host system, orby other devices in communication with the host system (e.g., via acommunication network). For example, the input data may come fromsensors, sensor-processing chips/modules/circuits, antennae, storagedevices, network interfaces, or any other source of data for the chip(s)9400 as described herein. The physical input interface 9402 may connectwith the source(s) of the inputs 9492 via wired or wireless connections.The inputs 9492 may include any type of data to which governance may beapplied. The inputs 9492 may also include data that may be stored instorage 9450, such as governance rules/configurations for the governancelibrary 9452, one or more digital twins for the digital twin library9454, one or more system specification(s) 9456, and/or one or moreintelligence module(s) 9458.

The output data 9494 transmitted from the physical output interface 9404may include report(s) indicating the status of governance functions(e.g., governance compliance and/or violations that may occur), dataindicating the functioning of generated models (e.g., as part of a modelvalidation process), instructions directed to othermodules/devices/systems to enforce compliance with governance standards,and/or the like. In embodiments, the outputs of the chip 9400 may betransmitted by the physical output interface 9404 to other chips,circuits, modules, and/or other components of a host system or anotherdevice in communication with the host system as described herein. Thephysical output interface 9404 may connect to these components via wiredor wireless connections.

In embodiments, the chip 9400 may include one or more of a governanceanalysis module 9410, a governance framework module 9420, and/or agovernance output module 9430. In embodiments, the governance analysismodule 9410 may comprise circuits 9412-9416 for receiving and processinginputs 9492 to determine governance applicability and to format theinput data for the application of governance. Additionally oralternatively, the chip 9400 may include a governance framework module9420 comprising circuits 9422-9426 for prioritizing governance, creatinggovernance models, and validating governance models. Additionally oralternatively, the chip 9400 may include a governance output module 9430comprising circuits 9432-9436 for executing, monitoring, and otherwiseprocessing a governance model, optimizing the model, and formattingresults for output. The functionalities of the various circuits of themodules 9410, 9420, and/or 9430 are described in more detail below.

The processing core(s) 9406 may comprise one or more processing core(s)that may be configured to perform any of the functions attributed to thechip 9400, either with or without the assistance of the various modules9410, 9420, and/or 9430. For example, the processing core(s) 9406 mayleverage and/or invoke various modules to perform various functionsdescribed herein. The processing core(s) 9406 may comprisegeneral-purpose and/or special-purpose processors. In embodiments, theprocessing core(s) 9406 may use serial, parallel, and/or otherprocessing techniques to accomplish the functions described herein.

Accordingly, the processing core(s) 9406 may perform functions inaddition to the functions provided by the various modules 9410, 9420,and/or 9430. For example, the processing core(s) may receive an outputof one module (e.g., data extracted by a data set analyzed by thegovernance analysis module 9410) and provide it as input to anothermodule (e.g., to the governance framework module 9420 and/or thegovernance output module 9430). The processing core(s) 9406 may alsoprocess the output of any of the module(s) to convert the output into adifferent format.

In embodiments, the processing core(s) 9406 may further operate to storeand/or retrieve data to/from storage 9450. For example, the processingcore(s) 9406 may store and retrieve governance configurations/data in agovernance library 9452 and/or digital twins in a digital twin library9454, may store and retrieve system specifications 9456, and/or maystore and retrieve intelligence module(s) 9458 for implementing thevarious functions described herein. In embodiments, the processingcore(s) may implement any of the functionalities of the intelligenceservice 8800 (as described with respect to FIG. 104 ) using theintelligence modules 9458 (which may include one or more of theartificial intelligence modules 8804 of FIG. 104 ).

The governance analysis module 9410 may receive and process input data9492 to determine whether and what type of governance may apply. Inembodiments, the input data analysis circuit 9412 may analyze the inputs9492 to detect conditions indicating that governance applies. Forexample, the input data may indicate a particular location that may beassociated with governance requirements (e.g., governance requirementsset by the owner of a property corresponding to the location, governancerequirements set by a particular state or other government entitycorresponding to the location, etc.). As another example, the input datamay include a particular data field, and one or more values of the datafield may indicate that governance applies. Additionally oralternatively, the input data analysis circuit may access the governancelibrary 9452, digital twins 9454, and/or system specifications 9456 inorder to determine whether one or more governance standards apply. Forexample, a governance library 9452 may indicate one or more conditionsin which governance standards apply, that certain governance standardsalways apply, and/or provide other rules, triggers, or conditionsindicating that governance standards apply. In embodiments, a digitaltwin may indicate that the input data 9492 relates to a device having aparticular state within the digital twin, and the particular state maybe associated with a particular set of governance standards. Similarly,a system specification 9456 may provide information about a systemcorresponding to the data and may indicate if/when governance applies tothe system. Thus, using one or more strategies including analyzing theinput data 9492 and/or data within storage 9450, the chip 9400 maydetermine that governance does or does not apply to input data receivedas inputs 9492.

In embodiments, the governance selection circuit 9414 may determinewhich of the identified governance requirements applies. For example,one or more governance rules related to safety, security, quality,regulatory, financial, or other standards may apply based on variousconditions as explained above, such as a location or other conditioncorresponding to the input data, a type of data received as input data,one or more values received as input data, data stored in storage 9450,and/or the like. The one or more conditions, triggers, values, or otherindications that governance requirements apply, as detected by the inputdata analysis circuit 9412, may each correspond to one or moregovernance requirements, which the governance selection circuit 9414 mayretrieve and select. In some cases, the governance selection circuit9414 may need to further analyze data (e.g., using intelligence modules9458) to determine which governance requirements apply. For example, thegovernance selection circuit 9414 may process the inputs 9492 using aneural network or other machine learned model to generate a prediction,and then based on the prediction may determine which governancerequirements apply. In embodiments, multiple intelligence module(s) 9458may be used to provide various types of AI analysis for governanceselection. In embodiments, the intelligence modules 9458 may include oneor more of the artificial intelligence modules 8804 of FIG. 104 .

In embodiments, the data analysis circuit 9416 may perform data analysisto determine and/or extract data to apply governance. For example, thedata analysis circuit 9416 may parse or otherwise analyze the inputs9492 to extract particular values to which governance applies and/or todetect particular values to which governance does not apply. Inembodiments, the data analysis circuit 9416 may generate one or moredata structures comprising the extracted data and format the datastructure so that governance standards may be generated and/or enforcedusing the data structure. The data analysis circuit 9416 may access anyof the data stored in storage 9450, which may specify how to detect datavalues to which governance applies for the governance requirementsselected by the governance selection circuit 9414.

The governance framework module 9420 may receive one or more selectedgovernance requirements from the governance analysis module 9410 and maydevelop and validate a model for applying the governance requirements tosets of data. In embodiments, the prioritization circuit 9422 may managemultiple and/or overlapping governance requirements by prioritizing thegovernance requirements, resolving conflicts between the governancerequirements, and/or the like. The prioritization circuit 9422 mayassign a priority to each of the governance requirements selected by thegovernance selection circuit 9414 (e.g., by retrieving an assignedpriority associated with each governance requirement from the governancelibrary 9452, by using one or more prioritization rules included in thegovernance library 9452, etc.). In embodiments, the prioritizationcircuit 9422 may detect whether any of the selected governancerequirements overlap or conflict. In some cases, the governancerequirements may overlap without causing a conflict, such as when afirst governance requirement requires a certain minimum standard, and asecond governance requirement requires a higher standard. In such acase, the prioritization circuit 9422 may determine that the higherstandard should be used in order to meet both sets of governancerequirements. In other cases, such as when the governance requirementsconflict, the prioritization circuit 9422 may determine to use one orthe other conflicting standards based on the priorities assigned to eachgovernance requirement.

In embodiments, the modeling circuit 9424 may generate a model based onthe prioritized governance requirements as determined by theprioritization circuit 9422. For example, if the highest prioritygovernance requirement is a set of safety requirements, then thegenerated model may initially check for safety violations or apply othersafety governance requirements. Then, if the second highest prioritygovernance requirement is a set of regulatory governance requirements,the model may, after enforcing safety governance, enforce regulatorygovernance. In some cases (e.g., due to conflicts), the model may omitcertain governance requirements from the model (e.g., a qualityrequirement that conflicts with a safety requirement). In this way, themodeling circuit 9424 may generate a model specifying a flow forenforcing governance on a data set. The modeling circuit 9424 may causethe generated model to reference various digital twins from digital twinlibrary 9454 that specify information about one or more environments,networks, systems, or the like, to retrieve various data that may benecessary for checking and enforcement.

In embodiments, the validation circuit 9426 may validate the generatedmodel, for example by testing it against test data provided by thegovernance library 9452. In some cases, the selected governancestandards may require certain validations (e.g., validation that themodel complies with safety requirements when processing data), and thusthe governance library may contain test data and/or target output(s) forvalidating that the model successfully complies with the correspondinggovernance requirement(s). Additionally or alternatively, the validationcircuit 9426 may test the generated model against a digital twin tosimulate its effect on one or more devices, networks, systems, etc. Insome cases, the simulated effect on the digital twin may be provided asan output 9494 (e.g., for analysis/approval at another device) beforedeploying the generated model to the governance output module 9430.

In embodiments, the governance output module 9430 may use the generatedmodel to process one or more inputs 9492 to enforce the governancestandards, may optimize the model based on varying conditions, and/ormay output the processed inputs, reports, and/or messages forcommunicating with other devices. The model processing circuit 9432 maycontinually process inputs 9492 (e.g., the inputs that were analyzed bythe governance analysis module 9410 as well as inputs received after thegovernance model is generated by the governance framework module 9420)as they are received, such that the governance model, once deployed, maybe used on new inputs. The model processing circuit 9432 may use themodel to monitor inputs 9492 and enforce the governance standards asspecified by the model. For example, the model processing circuit 9432may generate warnings and alarms, shut down or otherwise modify systems(e.g., if safety parameters have been exceeded),modify/transform/configure data to comply with governance, and/or thelike. In embodiments, to enforce the governance requirements, the chip9400 may send messages and/or instructions to other devices and systems.In these cases, the model optimization circuit 9434 may cause the outputand reporting circuit 9436 to send such messages and/or instructions, asexplained in more detail below.

In embodiments, the model optimization circuit 9434 may perform liveoptimization of the governance framework/model by continually monitoringvarying input conditions and data. For example, in response to a changein location or some other condition, a different set of governancerequirements may begin to apply. The model optimization circuit 9434 mayenforce this different set of governance requirements by causing thegovernance framework module 9420 to regenerate and/or modify the modelto prioritize the new governance requirements, update the modelaccordingly, and/or validate the updated model, as described above.Additionally or alternatively, the model optimization circuit 9434 maycontinually validate the output of the model processing circuit 9432 toensure that the model used by the model processing circuit 9432 isperforming appropriately. As described above, the model optimizationcircuit 9434 may perform the validation with reference to validationdata/requirements stored in the storage 9450.

In embodiments, the output and reporting circuit 9436 may transmitoutputs including data processed by the model processing circuit 9432,as well as messages and/or instructions to be sent to other modules,device, systems, etc. Accordingly, the chip 9400 may enforce governancerequirements by causing other devices to change state (e.g., turnoff/on) or otherwise perform governance actions. Additionally oralternatively, the output and reporting circuit 9436 may generatereports including results of the validations, reports indicating alertsor other noncompliance with governance, reports indicating thatgovernance conflicts, and the like, for review/analysis by other chips,modules, systems, or devices. The output and reporting circuit 9436 maycause any outputs to be transmitted as outputs 9494.

FIG. 113 illustrates a prediction, classification, and recommendationchip 9500, one or more of which may be used to perform one or moreprediction, classification, and/or recommendation functions as describedherein. The chip 9500 may be used by any value chain entity thatperforms prediction, classification, and/or recommendation. Inembodiments, a chip 9500 may use artificial intelligence (AI) and othertechniques to perform the prediction, classification, and/orrecommendation functions on input data from one or more components of ahost device incorporating the chip 9500 and/or other devices incommunication with the host device. In embodiments, the chip 9500 may beconfigured to analyze and classify incoming data according to a givenset of specifications, to develop and/or optimized predictive modelsaccording to a given set of specifications, and/or to providerecommended actions based on the data classifications and predictivemodeling according to a set of specifications.

In embodiments, the chip 9500 may be configured to receive variousinputs of any type, including media data such as images/video/audiodata, data sets including transaction data, biometric data, motioncapture data, pathology data, and/or other such data, and to analyzesuch data to determine further information (e.g., metadata) about theinput data, objects or entities appearing in the input data, and thelike. The chip 9500 may then classify the inputs, objects or entitiesappearing in the inputs, or the like using various classificationtechniques, as explained in detail below. The chip 9500 may output theclassifications as outputs 9594 for use by other modules, devices,systems, and the like.

In embodiments, the chip 9500 may develop one or more conditions for usein generating a predictive model. The conditions may be developed basedon the classifications. In other words, based on classifying certainobjects, entities, or groupings thereof, one or more conditions relatedto the objects, entities, or groupings thereof may be developed andselected for predictive analysis, in order to determine the effects ofvarious actions involving the objects, entities or groupings thereof.Then, the chip 9500 may generate and leverage a predictive model topredict the effects of an action involving the objects, entities, orgroupings thereof, and may further optimize the predictive model basedon updated data, as described in more detail below.

In embodiments, the chip 9500 may use various system specifications togenerate an action matrix comprising one or more actions, one or moredirect or indirect objects or other entities on which the actions may betaken, one or more action modifiers, and/or the like, in order todetermine a range of actions that may be taken related to variousentities. The chip 9500 may then analyze and decide which action(s) fromthe action matrix should be taken (e.g., using the generated predictivemodels), transmit outputs 9594 causing performance of the selectedactions, and provide feedback to improve the functioning of theclassification, prediction, and recommendation functions, as describedin more detail below.

In embodiments, the chip(s) 9500 can be modular component(s) that may beintegrated with a host system in various ways. For example, the chip(s)may be integrated with a mobile host system, a stationary host system,or any other host system that receives input data for prediction,classification, and/or recommendation tasks. To facilitate thismodularity, the chip(s) 9500 may be provided partially or completelywithin a housing (not shown) and may receive the inputs 9592 and/orprovide the outputs 9594 via electrical connectors, optical connectors,and/or wireless connectors (e.g., antennae, inductive coils, etc.).Additionally or alternatively, the chip(s) 9500 may be integrated withother circuits, processors, systems, etc., either on one or multiplesubstrates/chips.

The chip(s) 9500 may be and/or include one or more system-on-chips(SOCs), integrated circuits (ICs), application-specific integratedcircuits (ASICs), and/or the like, for providing the functionalityattributed to chip 9500 and/or any other functionality. For example, thechip 9500 may be provided as part of a SOC that also provides otherfunctions described herein. In general, the components of the chip 9500may comprise one or more general-purpose processing chips that areconfigured using software instructions or other code, and/or maycomprise special-purpose processing chips (e.g., ASICs) customized toperform the functions described herein.

Multiple chip(s) 9500 may be used to perform the functions describedherein. For example, multiple chip(s) 9500 may use serial, parallel,and/or other processing techniques to perform AI-assisted functions morequickly, to perform AI-assisted functions more efficiently by offloadingmore complex computations from one chip 9500 to another chip 9500 with abetter power source, and/or the like. As another example, one chip 9500may be used to provide a first AI-assisted function described herein,while another chip 9500 may be used to provide a second AI-assistedfunction based on the same inputs 9592.

In embodiments, the physical input interface 9502 receives one or moreinputs 9592 to the chip 9500 as described herein. The inputs 9592 may betransmitted to the physical input interface 9502 by other chips,circuits, modules, and/or other components of the host system, or byother devices in communication with the host system (e.g., via acommunication network). For example, the input data may come fromsensors, sensor-processing chips/modules/circuits, antennae, storagedevices, network interfaces, or any other source of data for the chip(s)9500 as described herein. The physical input interface 9502 may connectwith the source(s) of the inputs 9592 via wired or wireless connections.The inputs 9592 may include any type of data to which governance may beapplied. The inputs 9592 may also include data that may be stored instorage 9550, such as governance rules/configurations for the governancelibrary 9552, one or more digital twins for the digital twin library9554, one or more system specification(s) 9556, and/or one or moreintelligence module(s) 9558.

The output data 9594 transmitted from the physical output interface 9504may include one or more classifications, predictions, and/or recommendedactions, as well as one or more reports for providing information aboutthe inputs to the chip 9500, data generated by the chip 9500, thefunctioning of the chip 9500, and/or the like.

In embodiments, the chip 9500 may include one or more of aclassification module 9510, a prediction module 9520, and/or arecommendation module 9530. In embodiments, the classification module9510 may comprise circuits 9512-9516 for receiving andextracting/isolating data, analyzing the data, and classifying the data.Additionally or alternatively, the chip 9500 may include a predictionmodule 9520 comprising circuits 9522-9526 for developing and/orotherwise leveraging conditions (e.g., based on the classificationsprovided by the classification module 9510), generating predictionsusing predictive models, and optimizing the predictive models.Additionally or alternatively, the chip 9500 may include arecommendation module 9530 comprising circuits 9532-9538 for generatinga recommended action matrix, analyzing applied decision criteria (e.g.,to select one or more recommended actions), reporting and/or otherwisecarrying out the recommended action(s), and providing feedback data foruse by the various modules and/or circuits of the chip 9500. Thefunctionalities of the various circuits of the modules 9510, 9520,and/or 9530 are described in more detail below.

The processing core(s) 9506 may comprise one or more processing core(s)that may be configured to perform any of the functions attributed to thechip 9500, either with or without the assistance of the various modules9510, 9520, and/or 9530. For example, the processing core(s) 9506 mayleverage and/or invoke various modules to perform various functionsdescribed herein. The processing core(s) 9506 may comprisegeneral-purpose and/or special-purpose processors. In embodiments, theprocessing core(s) 9506 may use serial, parallel, and/or otherprocessing techniques to accomplish the functions described herein.

Accordingly, the processing core(s) 9506 may perform functions inaddition to the functions provided by the various modules 9510, 9520,and/or 9530. For example, the processing core(s) may receive an outputof one module (e.g., classification data generated by the classificationmodule 9510) and provide it as input to another module (e.g., to theprediction module 9520). The processing core(s) 9506 may also processthe output of any of the module(s) to convert the output into adifferent format.

In embodiments, the processing core(s) 9506 may further operate to storeand/or retrieve data to/from storage 9550. For example, the processingcore(s) 9506 may store and retrieve governance configurations/data in agovernance library 9552 and/or digital twins in a digital twin library9554, may store and retrieve system specifications 9556, and/or maystore and retrieve intelligence module(s) 9558 for implementing thevarious AI-assisted functions described herein. In embodiments, theprocessing core(s) may implement any of the functionalities of theintelligence service 8800 (as described with respect to FIG. 104 ) usingthe intelligence modules 9558 (which may include one or more of theartificial intelligence modules 8804 of FIG. 104 ).

The classification module 9510 may receive input data, isolate/extractthe input data, analyze the data, and classify the data. In embodiments,the data isolation circuit 9512 may receive input data 9592 and extractor otherwise isolate the input data prior to analysis. For example, theinput data 9592 may be one or more data streams or data sets comprisingimage/video data, transaction data, biometric data, diagnostic data, orany other type of data as described herein. The data isolation circuit9512 may isolate such data from a data stream/set (e.g., byidentification of the data for analysis, extraction of the data,conversion/re-formatting of the data, etc.). For example, the dataisolation circuit 9512 may extract images from video, convert speech totext, extract relevant data from a larger data set, and/or the like.

In embodiments, the analysis circuit 9514 may analyze the isolated dataand/or other data to determine information for classification. Forexample, the analysis circuit may perform image analysis on images toidentify one or more objects appearing in the images, may analyzetransaction data to determine transaction metadata (e.g., the identityof a sender/receiver, a type of transaction, etc.), may analyzebiometric data to determine personal metadata (e.g., an identity,demographic information, etc.), may analyze motions shown in video data(e.g., to determine movement, expression, and/or reaction information),may analyze diagnostic data (e.g., to determine abnormalities or otherconditions from diagnostic data sets), and/or the like. In some cases,the analysis circuit 9514 may leverage information stored in storage9550 to perform the analyses. For example, the analysis circuit 9514 mayuse various digital twins from digital twin library 9554 and/or systemspecifications 9556 to obtain information about various systemscorresponding to input data (e.g., to provide additional informationabout a device or other entity corresponding to input data, to allowinterpretation of input data, etc.), may use intelligence modules 9558to perform various analyses (e.g., a machine vision intelligence moduleto perform object recognition), and/or the like. Additionally oralternatively, the analysis circuit 9514 may structure the data forclassification by the classification circuit 9516.

In embodiments, the classification circuit 9516 may performclassification tasks on the isolated data and/or any additional datagenerated by the analysis circuit 9514. The classification circuit 9516may use one or more machine learning or otherwise AI-assisted techniques(e.g., regressions, naive Bayes, stochastic gradient descent, k-nearestneighbors, decision trees, random forests, etc.) to classify the data.For example, the classification circuit 9516 may classify objectsappearing in images (e.g., by identifying the type of other grouping ofobjects), may classify transaction data (e.g., by type of transaction,by whether the transaction is abnormal/suspicious/etc., by type of partyto the transaction, etc.), may classify people according to biometricdata (e.g., by demographics, by type of emotion, etc.), may classifymotion data (e.g., by reaction type), may classify diagnostic data(e.g., to identify pathologies or other abnormalities in individual orpopulation data), and/or the like. The classification circuit 9516 mayleverage unsupervised machine learning techniques to group the dataisolated by the data isolation circuit 9512 and/or generated by theanalysis circuit 9514, and/or may use supervised learning techniques(e.g., trained models that may be stored in storage 9550 as intelligencemodules 9558) for a particular task. Accordingly, the chip 9500 may beconfigured for a particular classification task by storing appropriateconfiguration data (e.g., trained models) in the storage 9550.

The prediction module 9520 may develop, leverage, and/or optimizeprediction models to generate predictions based on data received asinputs 9592 and/or one or more specifications 9556. In embodiments, thecondition development circuit 9522 may develop conditions that may beused to generate predictive models based on the classificationsperformed by the classification model. When the classification circuit9516 detects one or more classifications, the condition developmentcircuit 9522 may select one or more conditions related to theclassification to target using a predictive model. For example, based onthe classification circuit 9516 recognizing certain types of objects inimages, the condition development circuit 9522 may develop a targetvariable related to the detected type of object (e.g., anumber/amount/frequency of the object or other target variable that is afunction of the object) for use in development of a predictive model. Asanother example, based on the classification circuit 9516 recognizingcertain types of transactions in transaction data, the conditiondevelopment circuit 9522 may select as a target variable an estimate offuture transactions of the detected type. As another example, based onthe classification circuit 9516 recognizing certain types of behaviorsor demographics, the condition development circuit 9522 may develop atarget variable comprising an assessment of object or group behavior,security estimates (e.g., based on unsafe behavior), cognitiveassessments, and/or the like. As another example, based on theclassification circuit 9516 recognizing certain types of pathologies,the condition development circuit 9522 may develop a target variablecomprising an estimated spread of a pathology, a population change, acost of addressing the pathology, etc. In some cases, one or more storedsystem specifications 9556 may indicate which conditions are availableto target and/or should be targeted. Accordingly, the chip 9500 may beconfigured for a particular system/task/domain by storing particularsystem specifications 9556.

In embodiments, the predictive modeling circuit 9524 may use the targetvariable generated by the condition development circuit 9522 to train aprediction model for predicting the target variable based on the inputdata, data generated by the analysis circuit 9514, and/or classificationdata generated by the classification circuit 9516. In other words, thepredictive modeling circuit 9524 may use a training data set comprisingany of the aforementioned data to train the model to predict the targetvariable. The predictive modeling circuit 9524 may use variousAI-assisted learning techniques (e.g., neural networks, deep learning,etc.) to develop the model based on the selected target variable.

Additionally or alternatively, the predictive modeling circuit 9524 mayleverage the predictive model to generate predictions based on variousmodeling inputs. The modeling inputs may be derived from the inputs 9592(e.g., the input data that was used by the classification module 9510 asdescribed above and/or a new set of input data), the isolated/extractedinput data generated by data isolation circuit 9512, the data generatedby analysis circuit 9514, the classifications generated byclassifications 9516, etc. In other words, any of the data received asinputs 9592 and/or generated by the chip 9500 may be used as inputs tothe predictive model. The predictive modeling circuit 9524 may providethe various inputs to the predictive model to generate a prediction,which may comprise one or more discrete and/or continuous values (e.g.,predicted scores and/or classifications), one or more confidences, etc.

In embodiments, the predictive model optimization circuit 9526 mayoptimize the predictive model by updating the training data set,re-training the predictive model, selecting a different target variableand developing a new model, and/or the like. For example, the predictivemodel may periodically update the training data set and re-train themodel using new data that is received as inputs 9592 and/or generated byany of the analysis circuit 9514, classification circuit 9516, and/orpredictive modeling circuit 9524. Additionally or alternatively, thepredictive model optimization circuit 9526 may monitor the accuracy ofpredictions by monitoring input data 9592 and/or one or more digitaltwin(s) from digital twin library 9554 over time. For example, if thepredictive modeling circuit 9524 repeatedly predicts a future conditionwith a high confidence, but the predictive model optimization circuit9526 later determines that the predicted conditions do not occur, thepredictive model optimization circuit 9526 may causeupdating/modification of the training data set and/or trainingparameters and re-training of the predictive model to provide moreaccurate predictions.

In embodiments, the recommendation module 9530 may providerecommendations based on various specifications 9556, theclassifications generated by the classification module 9510, and/or thepredictions generated by the prediction module 9520. In embodiments, theaction matrix circuit 9532 may generate a matrix (e.g., an N-dimensionalarray, which may include a simple list) of potential actions that may betaken in relation to a particular task, system, or domain. For example,a system specifications 9556 may provide a first set of potentialactions, a second set of potential entities on which the actions may betaken, a third set of modifiers for the actions, etc., and the actionmatrix circuit 9532 may thus generate a matrix of potential actions thatmay be recommended. Additionally or alternatively, certain actions,entities, etc. may be automatically identified and added to an actionmatrix based on input data, analyses performed by the analysis circuit9514, classifications generated by the classification circuit 9516,and/or predictions generated by the predictive modeling circuit 9524.Additionally or alternatively, certain actions, entities, etc. may beautomatically kept out or removed from the action matrix based ongovernance data (e.g., from governance library 9552). Actions mayinclude instructions addressed to digital and/or real-world entities,such as instructions to be performed by humans, computing devices,systems, modules, etc.

In embodiments, the decision analysis circuit 9534 may analyze some orall of the actions of the action matrix in order to determine one ormore recommended actions. The decision analysis circuit 9534 mayleverage digital twin(s) in digital twin library 9554 to simulate theeffect of certain actions (which may involve, for example, using theprediction module 9520, an intelligence module 9558, and/or some otherresource to predict the effect of the action). Additionally oralternatively, the decision analysis circuit 9534 may use one or moregovernance requirements stored in governance library 9552 to determinethat certain actions violate governance requirements (e.g., because theyare unsafe or illegal) and/or that certain actions are required tocomply with governance requirements. As a first example, based ondetecting certain types of objects appearing in one or more images(e.g., as determined by classification module 9510) and predicting thatthe objects may reduce a target variable (e.g., as determined byprediction module 9520), the decision analysis circuit 9534 mayrecommend interacting with the objects to increase the target variable(e.g., by moving or otherwise interacting with the objects). As anotherexample, based on detecting certain types of transactions fromtransaction data (e.g., as determined by classification module 9510) andpredicting that the transactions may lead to a particular negativeoutcome (e.g., as determined by prediction module 9520), the decisionanalysis circuit 9534 may recommend preventing future similartransactions. As a third example, based on detecting certain types ofconditions from biometric or diagnostic data (e.g., as determined byclassification module 9510) and predicting that a particular pathologyor other condition is present (e.g., as determined by prediction module9520), the decision analysis circuit 9534 may recommend a particularintervention. As a fourth example, based on detecting certain types ofindividual and/or group behaviors (e.g., as determined by classificationmodule 9510) and predicting that conditions are becoming abnormal orunsafe (e.g., as determined by prediction module 9520), the decisionanalysis circuit 9534 may recommend shutting down particular locations,systems, or taking other remedial actions.

In embodiments, the recommended action and reporting circuit 9536 maycarry out the one or more recommended actions and/or cause transmissionof an output message (e.g., via outputs 9594) that may cause othermodule(s), device(s), system(s), etc. to carry out the recommendedactions. Additionally or alternatively, the recommended action andreporting circuit 9536 may generate reports that may includeclassifications, predictions, recommendations, and/or any of the otherdata received or generated by the chip 9500. The recommended action andreporting circuit 9536 may transmit the reports to other modules,devices, systems, etc., as outputs 9594.

In embodiments, the feedback circuit 9538 may monitor outcomesassociated with classifications, predictions, and/or recommended actionsto determine if the classifications and/or predictions were accurate, ifthe recommended actions had the desired/predicted impacts, and/or thelike. Accordingly, the feedback circuit 9538 may leverage one or moredigital twin(s) in the digital twin library 9554 to monitor one or moredevices, systems, environments, etc. In these embodiments, the digitaltwin(s) may be continuously updated by another component (e.g., asdescribed elsewhere herein) that keeps the digital twins updated formonitoring by the feedback circuit 9538. Based on the monitoredoutcomes, the feedback circuit 9538 may adjust (e.g., retrain) anymodels used by the classification module 9510, prediction module 9520,and/or recommendation module 9530.

Although the classification, prediction, and recommendation chip 9500may thus perform a wide variety of classification, prediction, andrecommendation tasks, including any of the classification, prediction,and recommendation tasks described herein, a few examples may be usefulto explain the flexibility and functionality of the classification,prediction, and recommendation chip 9500. According to a first example,the chip 9500 may be configured to automatically analyze and classifysatellite images (e.g., to recognize specific vegetation types, densityand location, animal population and movement, etc.), to providepredictions based on classified objects in the images (e.g., cropvaluations, fire hazard assessments, water allocations and prices,etc.), and to provide recommendations based on the classifications andpredictions (e.g., crop production adjustments, clearing of brush,increase of insurance reserves, reduction of water allocations, etc.).According to this first example, a wide variety of inputs 9592 may beused, including enterprise resource planning system inputs (e.g.,inventory, pricing, accounting, sales, employee information), customerrelationship management system inputs (e.g., customer data, paymentmethods, etc.), security system inputs (e.g., data access andmanagement, surveillance video, authentication data), inputs comprisingcrime statistics, police reports, cost of living reports, and the like.Additionally, system specifications 9556 in this example may indicatethat various actions may include increasing/reducing/maintaining storehours, products, or services provided, adjusting levels of security, andthe like. Moreover, the system specifications 9556 may include lists ofthe stores, products or services which may be adjusted, such that athree-dimensional action matrix indicating an action, a store, and anadjustment may be developed. According to a second example, the chip9500 may be configured to automatically analyze and classify financialtransactions (e.g., to recognize fraud or theft, types of purchases,contracts, customers, products, etc.), to provide predictions based onthe transaction data (e.g., demand response, fraud estimation andresponse, asset allocation, etc.), and to provide recommendations basedon the classifications and predictions (e.g., increasing production,reallocation inventory, investing in security and enforcement, adjustingprofit forecasts, redeploying assets, etc.). According to a thirdexample, the chip 9500 may be configured to automatically analyze andclassify biometric (e.g., to recognize faces, voice, gestures, identifygroups, evaluate emotions, etc.), to provide predictions based on thebiometric data (e.g., personal or group behavior, security, cognitiveassessments, etc.), and to provide recommendations based on theclassifications and predictions (e.g., health or psychologicalscreenings, security authentications/evaluations, etc.). According to afourth example, the chip 9500 may be configured to automatically analyzeand classify motion capture data (e.g., to classify behavior as normalor abnormal, safe or unsafe, etc.), to provide predictions based on themotion capture data (e.g., group behavior based on individual reactions,etc.), and to provide recommendations based on the classifications andpredictions (e.g., interventions, re-routing of group flow patterns,etc.). According to a fifth example, the chip 9500 may be configured toautomatically analyze and classify pathology data (e.g., to detectdiseases, population health, disease prevalence and spread, etc.), toprovide predictions based on the pathology data and classifications(e.g., disease spread, population changes, health care costs, etc.), andto provide recommendations based on the classifications and predictions(e.g., quarantines, allocation of medical resources, adjustment ofinsurance premiums, etc.).

Additive Manufacturing

FIGS. 114-121 describe various embodiments of an additive manufacturingplatform. In embodiments, an additive manufacturing platform may be astandalone system or may be integrated into a larger system, where theadditive manufacturing platform is a value chain entity. In embodiments,“additive manufacturing” refers to a collection of versatile fabricationtechniques for rapid prototyping and/or manufacturing of parts thatallow 3D digital models (CAD designs) to be converted to threedimensional objects by depositing multiple thin layers of material, suchas according to a series of two-dimensional, cross-sectional depositionmaps.

Accordingly, the term “additive manufacturing platform” used hereinencompasses a platform that prints, builds, or otherwise produces 3Dparts and/or products at least in part using an additive manufacturingtechnique. The additive manufacturing platform may encompasstechnologies like 3D printing, vapor deposition, polymer (or othermaterial) coating, epitaxial and/or crystalline growth approaches, andothers, alone or in combination with other technologies, such assubtractive or assembly technologies, enables manufacturing of athree-dimensional product from a design via a process of formingsuccessive layers of the product, with optional interim or subsequentsteps to arrive at a finished component or system. The design may be inthe form of a data source like an electronic 3D model created with acomputer aided design package or via 3D scanner. The 3D printing orother additive process then involves forming a first material-layer andthen adding successive material layers wherein each new material-layeris added on a pre-formed material-layer, until the entire designedthree-dimensional product is completed. The additive manufacturingplatform may be a stand-alone unit, a sub-unit of a larger system orproduction line, and/or may include other non-additive manufacturingfeatures, such as subtractive-manufacturing features, pick-and-placefeatures, coating features, finishing features (such as etching,lithography, painting, polishing and the like), two-dimensional printingfeatures, and the like. Further, the platform may includethree-dimensional additive manufacturing machines configured for rapidprototyping, three-dimensional printing, two-dimensional printing,freeform fabrication, solid freeform fabrication, and stereolithography;subtractive manufacturing machines including computer numericalcontrolled fabrication machines; injection molding machines and thelike.

FIG. 114 is a diagrammatic view illustrating an example environment ofan autonomous additive manufacturing platform 10110 according to someembodiments of the present disclosure. The platform operates within amanufacturing node 10100, which in turn is a part of a larger network ofvalue chain entities. The manufacturing node 10100 includes an additivemanufacturing unit 10102, such as a 3D printer for printing with metalmaterials, biocompatible materials, bioactive materials, biologicalmaterials, or other more conventional additive manufacturing materials,or other additive manufacturing type described herein, in the documentsincorporated herein by reference, or as understood in the art. Themanufacturing node 10100 may include, among other elements, apre-processing system 10104, a post-processing system 10106 and amaterial handling system 10108. The autonomous additive manufacturingplatform 10110 helps in automating and optimizing the digital productionworkflow leading to better outcomes at all stages of operation, frominitial design through printing and supply chain logistics to points ofsale, service and utilization of resulting outputs, among others. Inembodiments, user Interface 10112 receives input data from data sources10114 as well as design and modelling data from design and simulationsystem 10116. A data processing and intelligence component 10118 of theautonomous additive manufacturing platform 10110 runs artificialintelligence systems, such as involving machine learning or otheralgorithms, neural networks, expert systems, models and others, toprocess the input data and calculate an optimal set of processparameters for printing or other additive manufacturing. Process controlcomponent 10120 of the autonomous additive manufacturing platform 10110then adjusts one or more process parameters in real time and theadditive manufacturing unit 10102 uses these process parameters tocomplete the additive manufacturing process. In embodiments, finishingsystems 10121 at the manufacturing node 10128, such as subtractivesystems, assembly systems, additional processing systems, and the likemay undertake further processing, optionally in iterative sequences withadditive stages, resulting in a finished item (e.g., a part, component,or finished good). In embodiments, the resulting product is thenoptionally packaged at a packaging system 10122 and may be shipped,using a shipping system 10124 and one or more value chain network (VCN)entities 10126 right up to an end customer. In other embodiments, theadditive manufacturing platform 10110 and/or a set of additivemanufacturing units 10102 may comprise portable or otherwise mobileunits, such as handheld units, units equipped with robotic or otherautonomous mobility, and/or units positioned in or on vehicles,including general purpose vehicles and special purpose vehicles. In suchcases, actions from design through delivery may occur in parallel withmobility of the units 10102 and in coordination, by the additivemanufacturing platform 10110, with the location and mobility of othervalue chain network entities 10126. In one of many possible examples, aset of autonomously mobile 3D printing units may be coordinated topoints of service work, such as a set of home or business locations,where they may be configured to print tools, parts, or other items tosupport the service work, such as repairs or replacements. Inembodiments, additive manufacturing, including design generation, designreview, preprocessing, and printing steps, may commence while the unit10102 is in transit to the point of service. In another example, amobile autonomous additive manufacturing unit 10102 (either autonomous,semi-autonomous or with an operator) and packaging unit may completefinal steps of manufacturing in transit, such as by adding customizationelements (e.g., a final coating of a selected color, a customer-specificdesign element, or the like) in transit and optionally completing finalpackaging in transit. In embodiments, one or more components of theadditive manufacturing platform 10110 may be disposed in or integratedwith a smart container or a smart package, as described elsewhere hereinand in the documents incorporated by reference herein. In embodiments, aset of additive manufacturing units 10102 may be integrated into or witha set of robotic systems, such as mobile and/or autonomous roboticsystems. For example, the additive manufacturing unit 10102 may becontained within the housing or body of a robotic system, such as amulti-purpose/general purpose robotic system, such as one that simulateshuman or other animal species capabilities. Alternatively, oradditionally, the additive manufacturing unit 10102 may be configured todeliver additive layering from a nozzle that is disposed on an operatingend of a robotic arm or other assembly. In embodiments, multipleadditive manufacturing units 10102, or multiple nozzles, printheads orother working elements may be integrated with a single mobile,autonomous, and/or or multi-purpose robotic system, such as where oneadditive manufacturing unit 10102 is housed and prints/layers within thebody of the robotic system (such as in a chamber, such as vacuumchamber, pressurized chamber, heated chamber, or the like) and anotheradditive manufacturing unit 10102 prints/layers or otherwise operatesupon an external site, such as a target location of a machine, product,or the like, such as by a nozzle, printhead, or the like that isdisposed on an arm or similar element of the robot. In embodiments,multiple printing/layering elements are served by a common materialsource, such as of thermoplastic material. In embodiments, multiplematerial sources are available for internal and externalprinting/layering elements. In embodiments, an internal printing elementoperates within a chamber using materials that require control over theprinting environment, or operates on high-value production elements,such as parts that are intended for long-term use, such as metalmanufactured parts. In embodiments, the external working unit usesmaterials or does jobs that require other materials and/or have otherpurposes, such as production of disposable tools, grips, supports,fasteners and the like in support of a job, such as a repair orreplacement job, among many others. In embodiments, the externalprinting/layering unit is combined with a robotic arc welding unit, suchas to provide, in series or parallel, a set of printing/layering stepsand a series of arc welding steps to undertake a job on an externalsite, workpiece, or the like. In embodiments, an assembly may beprovided to encapsulate and/or shield an external working unit, such asa temporary chamber, balloon, tent, or other volume that isolates thearea where the nozzle, printhead, or the like will print, layer or thelike, optionally also encapsulating or shielding a workpiece or targetlocation for printing/layering within the same shielded/isolated spaceas the additive manufacturing element. In embodiments, theencapsulated/shielded area may be sealed to allow pressurization,depressurization, vacuum creation, introduction of materials fordeposition, and the like. In embodiments, the encapsulation/shieldingmay use an additively manufactured element, or combination thereof withanother element. In embodiments, an AI system 10212 may automate one ormore of the design, configuration, scheduling, coordination and/orexecution of a set of robotic jobs and a set of additive manufacturingjobs, such that the capabilities of an integrated mobile robotic andadditive manufacturing unit are coordinated across the various jobs intime (e.g., where an interior 3D printer or other additive manufacturingunit 10102 prints a tool, workpiece, part or the like for a later jobwhile the robotic unit performs a current job) and/or wherein jobs arecoordinated across a fleet or workforce of robotic units, additivemanufacturing units, and integrated combinations thereof (such as whereunits are matched to jobs according to locations, robotic capabilities,additive manufacturing capabilities, and other factors).

In embodiments, material handling systems 10108 provide storage,movement, control and handling of materials through the process ofmanufacturing and distribution. For example, the material handlingsystems 10108 may feed, orient, load/unload, or otherwise manipulatemetal materials, biocompatible materials, bioactive materials,biological materials, or other more conventional additive manufacturingmaterials in the manufacturing space. In embodiments, the materialhandling systems 10108 may be semi or fully automated and may includeone or more robotic units for material handling.

In embodiments, the material handling systems 10108 may include orintegrate with, optionally in the same housing, unit or system, amaterial capture and processing system 10127 for capturing material(such as recapturing unused material from jobs and/or capturingavailable material from a work site, such as from used, broken, ordefective items) and rendering the material suitable to use as a sourcematerial, such as by: (a) automatically analyzing an item to determineits compatibility for use as source material (e.g., by identifying it asa given type of metal, alloy, polymer or plastic, such as by machinevision, chemical testing, image-based testing, weighing the item, or thelike); (b) cleaning, filtering, disassembling, or otherwisepre-processing the item or material, such as to remove non-conformingmaterial; (c) rendering a solid item or material into a thermoplasticstate, such as by controlled heating, such as according to amaterial-specific heating profile; (d) filtering or otherwise treatingthe material, such as to remove defects; (e) storing the item in anappropriate vessel or form factor for later use, with appropriatereporting of capacity and availability, such as to a broader system formanaging jobs, including cooling and/or otherwise processing thematerial into a wire, powder, mesh, rod, filament or the like until theneed for a job arises; (f) delivering the item for additivemanufacturing operation; and/or (g) reporting on measures of recaptureand savings, including material cost savings, savings on recyclingcosts, and/or time savings. For example, in embodiments a broken partmay be melted down onsite and reprinted. For example, in embodiments amaterial that would otherwise be disposed of or recycled may be rendereduseful on site, without the need for reverse logistics. In embodiments,a common heating source is used, with alternate points of heating atdifferent temperatures, to render recaptured material into athermoplastic state and for preparing material for additivemanufacturing operations.

The value chain entities 10126 include various entities involved inproduction, supply, demand, distribution or supply chain environmentsincluding any of the wide variety of assets, systems, devices, machines,components, equipment, facilities, individuals or other entitiesmentioned throughout this disclosure or in the documents incorporatedherein by reference, such as, without limitation: machines and theircomponents (e.g., delivery vehicles, forklifts, conveyors, loadingmachines, cranes, lifts, haulers, trucks, loading machines, unloadingmachines, packing machines, picking machines, and many others, includingrobotic systems, e.g., physical robots, collaborative robots (e.g.,“cobots”), drones, autonomous vehicles, software bots and many others);workers (such as designers, engineers, process supervisors, supply chainmanagers, floor managers, demand managers, delivery workers, shippingworkers, barge workers, port workers, dock workers, train workers, shipworkers, distribution of fulfillment center workers, warehouse workers,vehicle drivers, business managers, marketing managers, inventorymanagers, cargo handling workers, inspectors, delivery personnel,environmental control managers, financial asset managers, securitypersonnel, safety personnel and many others); suppliers (such assuppliers of goods and related services of all types, componentsuppliers, ingredient suppliers, materials suppliers, manufacturers, andmany others); customers (including consumers, licensees, businesses,enterprises, value added and other resellers, retailers, end users,distributors, and others who may purchase, license, or otherwise use acategory of goods and/or related services); retailers (including onlineretailers and others such as in the form of eCommerce sites,conventional bricks and mortar retailers, pop-up shops and the like);value chain processes (such as shipping processes, hauling processes,maritime processes, inspection processes, hauling processes,loading/unloading processes, packing/unpacking processes, configurationprocesses, assembly processes, installation processes, quality controlprocesses, environmental control processes (e.g., temperature control,humidity control, pressure control, vibration control, and others),border control processes, port-related processes, software processes(including applications, programs, services, and others), packing andloading processes, financial processes (e.g., insurance processes,reporting processes, transactional processes, and many others), testingand diagnostic processes, security processes, safety processes,reporting processes, asset tracking processes, and many others);wearable and portable devices (such as mobile phones, tablets, dedicatedportable devices for value chain applications and processes, datacollectors (including mobile data collectors), sensor-based devices,watches, glasses, hearables, head-worn devices, clothing-integrateddevices, arm bands, bracelets, neck-worn devices, AR/VR devices,headphones, and many others); a wide range of operating facilities (suchas loading and unloading docks, storage and warehousing facilities,vaults, distribution facilities and fulfillment centers, air travelfacilities (including aircraft, airports, hangars, runways, refuelingdepots, and the like), maritime facilities (such as port infrastructurefacilities (such as docks, yards, cranes, roll-on/roll-off facilities,ramps, containers, container handling systems, waterways, locks, andmany others), shipyard facilities, floating assets (such as ships,barges, boats and others), facilities and other items at points oforigin and/or points of destination, hauling facilities (such ascontainer ships, barges, and other floating assets, as well asland-based vehicles and other delivery systems used for conveying goods,such as trucks, trains, and the like); items or elements factoring indemand (i.e., demand factors) (including market factors, events, andmany others); items or elements factoring in supply (i.e., supplyfactors)(including market factors, weather, availability of componentsand materials, and many others); logistics factors (such as availabilityof travel routes, weather, fuel prices, regulatory factors, availabilityof space (such as on a vehicle, in a container, in a package, in awarehouse, in a fulfillment center, on a shelf, or the like), and manyothers); pathways for conveyance (such as waterways, roadways, airtravel routes, railways and the like); robotic systems (including mobilerobots, cobots, robotic systems for assisting human workers, roboticdelivery systems, and others); drones (including for package delivery,site mapping, monitoring or inspection, and the like); autonomousvehicles (such as for package delivery); software platforms (such asenterprise resource planning platforms, customer relationship managementplatforms, sales and marketing platforms, asset management platforms,Internet of Things platforms, supply chain management platforms,platform as a service platforms, infrastructure as a service platforms,software-based data storage platforms, analytic platforms, artificialintelligence platforms, and others); and many others.

The manufacturing node 10100 may also connect to other nodes like amanufacturing node 10128 through connectivity facilities so as toconstitute a distributed manufacturing network 10130. Also, thedifferent systems within the manufacturing node 10100 including theadditive manufacturing unit 10102, the pre-processing system 10104, thepost-processing system 10106, the material handling system 10108, theautonomous additive manufacturing platform 10110, the user interface10112, the data sources 10114 and the design and simulation system 10116as well as the different parts and products being printed may bereferred to as distributed manufacturing network entities.

In embodiments, connectivity facilities include various connectivityfacilities described throughout this disclosure and the documentsincorporated by reference herein, including network connections(including various configurations, types and protocols for fixed andwireless connections), Internet of Things devices, edge devices,routers, switches, access points, repeaters, mesh networking systems,interfaces, ports, application programming interfaces (APIs), brokers,services, connectors, wired or wireless communication links,human-accessible interfaces, software interfaces, micro-services, SaaSinterfaces, PaaS interfaces, IaaS interfaces, cloud capabilities, or thelike by which data or information may be exchanged between systems orsub-systems of the autonomous additive manufacturing platform 10110, aswell as with other systems, such as distributed manufacturing networkentities or external systems, such as cloud-based or on-premisesenterprise systems (e.g., accounting systems, resource managementsystems, CRM systems, supply chain management systems and many others).In embodiments, connectivity facilities use, include, or are integratedwith artificial intelligence or autonomous capabilities as describedherein and/or in the documents incorporated herein by reference, such asenabling self-organization or self-configuration of connectivity, datastorage, computation, data processing, packet routing, data filtering,quality-of-service, error correction, packet security, sessionmanagement, and the like. In embodiments, the additive manufacturingunit 10102 may incorporate a wireless mesh network node, such as an RFrepeater, optionally using software-defined bandpass filtering, suchthat a set of such additive manufacturing units 10102 may operate as acoordinated mesh on a defined network infrastructure (including physicaland/or virtual network resources). In embodiments, the additivemanufacturing unit 10102 may include network coding system forcontrolling the utilization of a data path between the additivemanufacturing unit 10102 and other additive manufacturing units 10102and/or to control the utilization of the data path between the additivemanufacturing unit 10102 and various edge, cloud, on-premises,telecommunications network and other information technology systems.

The additive manufacturing unit 10102 may be any suitable type ofprinter that executes any suitable type of 3D printing process, or anyother type of unit that executes another additive manufacturing process.Various different types of additive manufacturing units 10102 and 3Dprinting processes are discussed below for purposes of example. Thedisclosure, however, is not limited to the 3D printing processesdescribed below.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute Fused Deposition Modeling (FDM)™ process (also known as, forexample, Fused Filament Fabrication™). The process of FDM may involve asoftware process which may process an input file, such as an STL(stereolithography) file. An object may be produced by extruding smallbeads of, for example, thermoplastic material to form layers as thematerial hardens immediately after extrusion from a nozzle. Extrusion isthe 3D printing technique where the material, such as a polymer, metal(including alloys), or the like, is pushed in fluid form through a tubeand into a moving nozzle which extrudes the material to a targetlocation where the material subsequently hardens in place. By accuratelymoving the extruder either continuously or starting and stopping atextremely fast speeds the design is built layer by layer. The sourcematerial is typically supplied and stored in solid form, such as in afilament or wire that is wound in a coil and then unwound to supplymaterial to a heating element to render the material into athermoplastic state and an extrusion nozzle which can control the flowof the material between an “off” state and a maximal flow state. Aworm-drive, or any other suitable drive system, may be provided to pushthe filament into the nozzle at a controlled rate. The nozzle is heatedto melt the material. The thermoplastic materials are heated past theirstate transition temperature (from solid to fluid) and are thendeposited by an extrusion head. The nozzle can be moved in bothhorizontal and vertical directions, such as by a numerically controlledmechanism. In embodiments, the nozzle may follow a tool-path that iscontrolled by a computer-aided manufacturing (CAM) software package, andthe object is fabricated layer-by-layer, such as from the bottom up.

In embodiments, the additive manufacturing unit 10102 may includemultiple source materials and multiple extrusion nozzles (and supportingcomponents for the same, such as for movement and positioning), such asto allow (a) rapid switching between source materials, such asfacilitated by a valve set, such as a high-pressure valve set, and/or(b) simultaneous extrusion by multiple nozzles, such as to enablesimultaneous layering at different points of work on an item. Inembodiments, the additive manufacturing unit 10102 enables voxelatedsoft matter printing and/or metal printing via multi-material,multi-nozzle printing, with high-speed switching between materials,e.g., at speeds of 50 times per second or faster.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute an electron beam freeform fabrication (EBFFF) processes. TheEBFFF process may utilize electron beam welding technology to createmetallic parts. In embodiments, with the EBFFF method, metallic preformscan be manufactured from computer-generated 3D drawings or models. Thedeposition path and process parameters may be generated frompost-processing of a virtual 3D model and executed by a real-timecomputer control. The deposition takes place in a vacuum environment. Awire may be directed toward the molten pool and melted by a focusedelectronic beam. Different parts of the object to be fabricated arebuilt up layer by layer by moving the electronic beam and wire sourceacross a surface of underlying material referred to as a substrate. Thedeposit solidifies immediately after the electron beam has passed.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute direct metal laser sintering process (DMLS). DMLS process mayinvolve a laser as a power source to sinter powdered material such as ametal at points in space defined by a 3D model thus binding the materialtogether to create a solid structure. The DMLS process may involve theuse of a 3D CAD model whereby a file, such as an .stl file, is createdand sent to the software of the additive manufacturing unit 10102. TheDMLS-based 3D printer may use a high-powered fiber optic laser. Themetal powder is fused into a solid part by melting it locally using thefocused laser beam. Object parts are built up additively layer by layer.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute a selective laser melting (SLM) process. The SLM process uses3D CAD data as a digital information source and energy in the form of ahigh-power laser beam to create 3D metal parts by fusing fine metallicpowders together. The process involves slicing of the 3D CAD file datainto layers to create a 2D image of each layer. Thin layers of atomizedfine metal powder are evenly distributed using a coating mechanism ontoa substrate plate that is fastened to an indexing table that moves inthe vertical (Z) axis. This takes place inside a chamber containing atightly controlled atmosphere of inert gas such as argon. Once eachlayer has been distributed, each 2D slice of the geometry is fused byselectively applying the laser energy to the powder surface, bydirecting the focused laser beam using two high frequency scanningmirrors in the X- and Y-axes. The laser energy permits full melting ofthe particles to form solid metal. The process is repeated layer afterlayer until the part is complete. In embodiments, the SLM process may bea multi-scanner and/or multi-laser SLM process, such as enablingsimultaneous action across multiple scans and/or multiple target pointsof laser melting work.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute a selective heat sintering process. The process may involve athermal printhead to apply heat to layers of powdered source material torender it to a thermoplastic state. When a layer is finished, the powderbed of source material moves down, and an automated roller adds a newlayer of material, which is sintered to form the next cross-section ofthe object. Power bed printing may refer to a technique where one ormore powders, typically a metal powder, are connected via variousmethods such as lasers or heat in order to rapidly produce the endproduct. Typically, it is done by either having an area filled withpowder and only connecting the design areas of the powder while layer bylayer removing the rest, or by adding powder layer-by-layer whilesimultaneously connecting it. Similar to light polymerization, powderbed printing is significantly faster than other types of 3D printing. Inembodiments, the additive manufacturing unit 10102 may employ multiplepowder bed/roller subsystems, thereby enabling simultaneous work ondifferent target points of work and/or multi-material powder bedapplications that allow switching between materials.

In embodiments, the additive manufacturing unit 10102, of various typesdescribed herein, may combine materials to produce an output comprisinga composite of materials, such as to combine favorable properties (e.g.,mechanical properties) of two materials to provide benefits that surpassthose of a single material. In embodiments, composite materials producedin or by the additive manufacturing units 10102 may comprisefunctionally graded materials (FGMs), such as where two materials arejoined with a graded interface that avoids a distinct boundary betweenthe materials. This may distribute thermal and/or mechanical stressesthat result from different material properties over a largervolume/space, thereby mitigating issues like cracking and breaking thatoccur with non-graded composite materials.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute a selective laser sintering process. The process of selectivelaser sintering (SLS) involves a laser used to melt a flame-retardantplastic powder, which then solidifies to form the printed layer. Inembodiments, the additive manufacturing unit 10102 may be configured toexecute a plaster-based 3D printing processes. In embodiments, theadditive manufacturing unit 10102 may be configured to execute alaminated object manufacturing process. In this process, layers ofadhesive-coated paper, plastic, or metal laminates may be successivelyglued together and cut to shape with a knife or laser cutter. After theobject is fabricated by the additive manufacturing unit 10102,additional modifications may be done by machining or drilling afterprinting. In embodiments, the selective laser sintering (SLS) involvesmultiple lasers, thereby allowing for switching and/or simultaneous workon different target locations and/or different material types.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute stereo-lithography (SLA) processes. The process may employ aresin, such as from a vat of liquid ultraviolet curable photopolymermaterial, and an ultraviolet laser to build layers one at a time. Foreach layer, the laser beam traces a cross-section of the part pattern onthe surface of the liquid resin. Exposure to the ultraviolet laser lightcures and solidifies the pattern traced on the resin and joins it to thelayer below. In embodiments, the SLA process may involve multiple UVlasers, allowing for switching and/or simultaneous work on differenttarget locations and/or different material types.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute digital light processing (DLP) methods. Digital lightprocessing uses a projector to project an image of a cross-section of anobject into a vat of photopolymer (light reactive plastic). The lightselectively hardens only the area specified in that image. A printedlayer is then repositioned to leave room for unhardened photopolymer tofill the newly created space between the print and the projector.Repeating this process builds up the object one layer at a time. Inembodiments, multiple DLP sources deliver light to different locations,allowing for switching and/or simultaneous work on different targetlocations within the light reactive plastic material.

In embodiments, the additive manufacturing unit 10102 may be configuredto execute light polymerization methods. In this process, drops of aliquid plastic are exposed to a laser beam of ultraviolet light. Duringthis exposure, light converts the liquid into a solid. Lightpolymerization may employ a technique where a rising or falling layer oflight-sensitive polymer is subjected to the type of light which causesit to harden in changing areas over time as it rises or falls and/or atechnique where a moving (e.g., laser) light source is targeted todifferent locations where liquid polymer/plastic material is positioned.This causes these areas of the polymer to harden, and once the desiredshape is created, the remaining liquid polymer that did not harden isremoved, leaving the finished product. Light polymerization is usefulbecause of how fast the final product completes with some types workingup to a hundred times faster, or more, than other 3D printing methodsfor some designs.

In embodiments, the additive manufacturing unit 10102 may involve theuse of an inkjet type printhead to deliver a liquid or colloidal bindermaterial to layers of a powdered build material. The printing techniquemay involve applying a layer of a powdered build material to a surface,such as using a roller. After the build material is applied to thesurface, the printhead delivers the liquid binder to predetermined areasof the layer of material. The binder infiltrates the material and reactswith the powder, causing the layer to solidify in the printed areas by,for example, activating an adhesive in the powder. After the firstcross-sectional portion is formed, the steps are repeated, andsuccessive cross-sectional portions are fabricated until the finalproduct is formed.

In embodiments, the methods performed by the additive manufacturing unit10102 may involve deposition of successive layers of a build material ona rotary build table and deposition of a liquid in a predeterminedpattern on each successive layer of the build material to form a 3Dobject.

In embodiments, the additive manufacturing unit 10102 may incorporatemultiple types of additive manufacturing capabilities among thosedescribed herein or understood by those of ordinary skill in the art,thereby forming a hybrid additive manufacturing unit. In embodiments,hybrid additive manufacturing units may further integrate othermanufacturing capabilities, such as subtractive techniques, assemblysystems, handling systems, finishing systems, and the like. Inembodiments, a hybrid additive manufacturing unit may integrate injectdelivery of a colloidal binder material with a liquid polymerizationtechnique.

In embodiments, the platform 10110 may provide 3D printed products thatconform to a body part/anatomy of the user including wearables likeeyewear, footwear, earwear and headgear. Conformance may, inembodiments, be based on a scan of a body part or anatomical feature,such as a laser or other structured light scan, a MRI, EEG, computedtomography, ultrasound or other imaging scan, or the like. A 3D topologyfor the anatomical feature may be used as an input source for generationby a CAD system or other design system (which may be linked to orintegrated into an additive manufacturing platform) of a design foradditive manufacturing. The design may be configured to produce ananatomy-compatible item that conforms well to anatomy (such as ahearable unit that fits the inner ear, headgear that fits the head, abrace that fits a joint, or the like) and/or an item that is intended toreplace a part of the anatomy, such as a prosthetic.

In embodiments, the platform 10110 has the capability to self-start andself-power.

In embodiments, the platform 10110 has a built-in recycling capabilitywherein scrap parts may be automatically returned to the productionprocess and support materials and excess powders may be returned to theproduction process.

FIG. 115 is a schematic illustrating an example implementation of theautonomous additive manufacturing platform for automating and optimizingthe digital production workflow for additive manufacturing (e.g., metalmanufacturing) according to some embodiments of the present disclosure.

The autonomous additive manufacturing platform 10110 includes a datacollection and management system 10202, a data storage system 10204 anda data processing system 10206. Manufacturing workflow managementapplications 10208 manage the various workflows, events and applicationsrelated to printing and supply chain including monitoring, inventoryaggregation, queue management, storage management, production reporting,production analysis and so on.

The data collection and management system 10202 collects and organizesdata collected from various data sources including real time datacollected from a set of sensors. Some examples of sensors providing dataas input to the data collection and management system 10202 include apower and energy sensor, mass sensor, location sensor, temperaturesensor, humidity sensor, pressure sensor, viscosity sensor, flow sensor,chemical/gas sensor, strain gauge to measure, image capture/camera,video capture, thermal imaging, hyperspectral imaging, sound sensor andair quality sensor.

The data storage system 10204 may store a wide range of data types usingvarious storage media, data architecture and formats including but notlimited to: entity or asset data (such as part profile, product profile,printer profile), state data (such as indicating a state, conditionstatus, or other indicator with respect to any asset, entity,application, components or elements of the platform 10110), user data(including identity data, role data, task data, workflow data, healthdata, performance data, quality data and many other types), event data(such as with respect to any of a wide range of events, includingoperational data, transactional data, workflow data, maintenance data,and many other types of data that includes or relates to events thatoccur within the platform 10110 or with respect to one or moreapplications, including process events, financial events, transactionevents, output events, input events, state-change events, operatingevents, workflow events, repair events, maintenance events, serviceevents, damage events, replacement events, refueling events, rechargingevents, shipping events, supply chain events, and many others); claimsdata (such as data relating to product liability, general liability,injury and other liability claims and claims data relating to contracts,such as supply contract performance claims, product deliveryrequirements, warranty claims, indemnification claims, deliveryrequirements, timing requirements, milestones, key performanceindicators and others); accounting data (such as data relating tocompletion of contract requirements, satisfaction of bonds, payment ofduties and tariffs, and others); and risk management data (such asrelating to parts or products supplied, amounts, pricing, delivery,sources, routes, customs information and many others), among many otherdata types associated with the platform 10110.

In embodiments, the data storage system 10204 may store data in adistributed ledger, digital thread or the like, such as for maintaininga serial or other record of an entity or asset over time, including apart or products or any other asset or entity described herein.

The data processing system 10206 includes an artificial intelligencesystem 10212, such as a machine learning system 10210. The machinelearning system 10210 may define a machine learning model 213 forperforming analytics, simulation, decision making, and predictiveanalytics related to data processing, data analysis, simulationcreation, and/or simulation analysis of one or more of assets orentities of the distributed manufacturing network 10130 of FIG. 114 . Inembodiments, the platform 10110 may include a set of artificialintelligence systems 10212 (including any of the types described hereinor in the documents incorporated herein by reference) that areconfigured (a) to operate on a set of inputs and/or a set ofoptimization factors to automatically select a suitable type of additivemanufacturing for a design/job; (b) to automatically discover a set ofavailable additive manufacturing units 10102 (optionally includingsingle-type units and/or hybrid type units), (c) to automatically selecta set of units 10102 to perform an additive manufacturing job; (d) toautomatically schedule a set of additive manufacturing units 10102 toperform a set of additive manufacturing jobs; (e) to automaticallyconfigure a selected set of additive manufacturing units 10102 toundertake a set of additive manufacturing jobs using a set of designsprovided by the set of artificial intelligence system; and/or (f) toautomatically configure logistics and delivery of a set of outputs froma set of additive manufacturing units. In embodiments, the set of inputsmay include locations and types of available additive manufacturingunits 10102, current job schedules for additive manufacturing units,cost factors (such as material costs, energy costs, costs of ITresources, costs of labor, pricing for additive manufacturing services,and others), design inputs (such as functional requirements regardingstrength, flexibility, resilience, temperature tolerance, straintolerance, resistance to wear, water resistance, stress tolerance,weight bearing, tensile strength, load bearing, and many others), aswell as compatibility factors (including shape compatibility,biocompatibility, chemical compatibility, environmental compatibility,and others). Optimization factors may include aesthetic factors,compatibility factors (as noted above), economic factors (such asmarginal cost, total cost, profitability, price, brand impact, andothers), timing factors (such as for coordination with workflows andactivities, including various ongoing manufacturing, service,maintenance, marketing, delivery and/or logistics processes),prioritization factors, and many others. In embodiments, the artificialintelligence system of the platform 10110 is trained based on a trainingset of data that includes expert interactions with a set of additivemanufacturing projects that involve various types of additivemanufacturing options. In embodiments, the AI system is trained based onoutcome factors, such as product quality and/or product defect outcomes,economic outcomes, on-time completion outcomes, and the like, such asinvolving deep learning, supervised learning and/or semi-supervisedlearning. In embodiments, the AI system is distributed between theadditive manufacturing units 10102 and a host system, such as acloud-based system. In embodiments, the AI system is integrated into theadditive manufacturing unit 10102. In embodiments, the AI system isdistributed across a set of additive manufacturing units 10102, such asa mesh or network of additive manufacturing unit 10102 nodes, such thatthe above capabilities are coordinated across the units, such as byself-configuration of the units 10102 in coordination with other units,such as a fleet of additive manufacturing units 10102 owned by anenterprise and/or co-operated and/or shared by a set of users (such asin an “additive manufacturing as a service” system). As one exampleamong many possible examples, the AI system of the platform 10110 maytake a set of design requirements, such as functional requirements,generate a set of designs that satisfy the functional requirements,determine the optimal combination of additive manufacturing types toproduce each set of designs, find and compare available additivemanufacturing units for each combination (such as using economic factorsand other factors), and select, configure and schedule units toundertake the design. For example, among many possibilities across awide range of product categories, the AI system may take functionalrequirements for a customized wearable device for a latex-allergicindividual user that meets a design requirement of using biocompatible,waterproof materials, while being capable of withstanding impacts andbending, in a color that matches the customers exact preference from alarge palette of colors. The AI system may automatically generate aninstruction set for producing the wearable device using acombination/hybrid of light polymerization (operating on a non-latexpolymer) for components of the wearable that will touch the user and aDMLS process for interior metal/alloy components. The AI system may thenfind available units, such as different units or an integrated/hybridunit, schedule the units to undertake jobs (e.g., to fit a targeteddelivery time), configure the units, send the jobs and scheduledelivery. Thus, the AI system may automatically manage the design,generation and delivery, through use of a set of additive manufacturingunits, a highly customized product based on customer specific designrequirements, including health requirements, physical configurationrequirements, economic factors, and preferences, among many others.

In embodiments, the AI system is implemented as the intelligence layer140 that receives requests from a set of intelligence layer clients andresponds to such request by providing intelligence services to suchclients (e.g., a decision, a classification, a prediction or the like).

In embodiments, the machine learning model 10213 is an algorithm and/orstatistical model that performs specific tasks without using explicitinstructions, relying instead on patterns and inference. The machinelearning model 10213 may build one or more mathematical models based ontraining data to make predictions and/or decisions without beingexplicitly programmed to perform the specific tasks. The machinelearning model 10213 may receive inputs of sensor data or other data astraining data, including event data and state data related to one ormore of the entities or assets, or other inputs noted above orthroughout this disclosure. The sensor data input to the machinelearning model 10213 may be used to train the machine learning model10213 to perform the analytics, simulation, decision making, and/orpredictive analytics relating to the data processing, data analysis,simulation creation, and/or simulation analysis of the one or more ofthe distributed manufacturing network entities or assets. The machinelearning model 10213 may also use input data from a user or users of theautonomous additive manufacturing platform 10110. In embodiments, themachine learning model 10213 may use the input data and sensor data todetermine an optimal set of process parameters for 3D printing of a partby the additive manufacturing unit 10102. The machine learning model10213 may include an artificial neural network, a decision tree, alogistic regression model, a stochastic gradient descent model, a fuzzyclassifier, a support vector machine, a Bayesian network, a hierarchicalclustering algorithm, a k-means algorithm, a genetic algorithm, anyother suitable form of machine learning model, or a combination thereof.The machine learning model 10213 may be configured to learn throughsupervised learning, unsupervised learning, reinforcement learning,self-learning, feature learning, sparse dictionary learning, anomalydetection, association rules, a combination thereof, or any othersuitable algorithm for learning.

In embodiments, the artificial intelligence system 10212 may define adigital twin system 10214 to create a digital replica or digital twin ofone or more of the distributed manufacturing network entities. Thedigital twin of the one or more of the distributed manufacturing networkentities may use substantially real-time sensor data to provide forsubstantially real-time virtual representation of the distributedmanufacturing network entities and for simulation of one or morepossible future states of the one or more distributed manufacturingnetwork entities. The digital twin exists simultaneously with the one ormore distributed manufacturing network entities being replicated(physical twin) and may be updated continuously based on sensor data,test and inspection results, conducted maintenance, modifications etc.to reflect the current condition or parameter values of the one or moredistributed manufacturing network entities. The digital twin providesone or more simulations of both physical elements and characteristics ofthe one or more distributed manufacturing network entities beingreplicated and the dynamics thereof, in embodiments throughout thelifecycle of the one or more distributed manufacturing network entitiesbeing replicated. The digital twin may provide a hypothetical simulationof the one or more distributed manufacturing network entities, forexample during a design phase before the one or more entities aremanufactured or fabricated, or during or after construction orfabrication of the one or more entities by allowing for hypotheticalextrapolation of sensor data to simulate a state of the one or moredistributed manufacturing network entities, such as during high stress,after a period of time has passed during which component wear may be anissue, during maximum throughput operation, after one or morehypothetical or planned improvements have been made to the one or moredistributed manufacturing network entities, or any other suitablehypothetical situation. In embodiments, the machine learning model 10213may automatically predict hypothetical situations for simulation withthe digital twin, such as by predicting possible improvements to the oneor more distributed manufacturing network entities, predicting when oneor more components of the one or more distributed manufacturing networkentities may fail, and/or suggesting possible improvements to the one ormore distributed manufacturing network entities, such as changes toparameters, arrangements, components, or any other suitable change tothe distributed manufacturing network entities.

The digital twin allows for simulation of the one or more distributedmanufacturing network entities during both design and operation phasesof the one or more distributed manufacturing network entities, as wellas simulation of hypothetical operation conditions and configurations ofthe one or more distributed manufacturing network entities. The digitaltwin allows for analysis and simulation of the one or more distributedmanufacturing network entities, by facilitating observation andmeasurement of nearly any type of metric, including temperature,pressure, wear, light, humidity, deformation, expansion, contraction,deflection, bending, stress, strain, load-bearing, shrinkage, in, on,and around each of the one or more distributed manufacturing networkentities. The insights gained from analysis and simulation using digitaltwins may be passed onto the design or manufacturing processes forimprovement of these processes.

In embodiments, the machine learning model 10213 may process the sensordata including the event data and the state data to define simulationdata for use by the digital twin system 10214. The machine learningmodel 10213 may, for example, receive state data and event data relatedto a particular distributed manufacturing network entity and perform aseries of operations on the state data and the event data to format thestate data and the event data into a format suitable for use by thedigital twin system 10214 in creation of a digital replica of thedistributed manufacturing network entity. For example, one or moredistributed manufacturing network entities may include a product beingmanufactured by the additive manufacturing unit 10102. The machinelearning model may collect data from one or more sensors positioned on,near, in, and around the product. The machine learning model may performoperations on the sensor data to process the sensor data into simulationdata and output the simulation data to the digital twin system 10214.The digital twin system 10214 may use the simulation data to create oneor more product twins 10215, the simulation including for examplemetrics including temperature, wear, speed, rotation, and vibration ofthe product and parts thereof. The simulation may be a substantiallyreal-time simulation, allowing for a user of the platform 10110 to viewthe simulation of the product, metrics related thereto, and metricsrelated to parts thereof, in substantially real time. The simulation maybe a predictive or hypothetical situation, allowing for a user of theplatform 10110 to view a predictive or hypothetical simulation of theproduct, metrics related thereto, and metrics related to componentsthereof.

In embodiments, the machine learning model 10213 and the digital twinsystem 10214 may process sensor data and create a digital twin of a setof distributed manufacturing network entities to facilitate design,real-time simulation, predictive simulation, and/or hypotheticalsimulation of a related group of distributed manufacturing networkentities.

In embodiments, a control system 10216 in the data processing system10206 may adjust process parameters of the 3D printing process inreal-time based on the simulations.

In embodiments, a distributed manufacturing network entity, such as theadditive manufacturing unit 10102 or the platform 10110, may, optionallyautomatically, generate a set of digital twins of a set of manufactureditems, such as products, components, parts, or the like. In embodiment,the digital twin of a manufactured item generated by the additivemanufacturing unit 10102 or the platform 10110 may include, link to, beenriched by, and/or integrate with, among other things: (a) aninstruction set according to which an item was additively manufactured,such as including shape information, material layering information,functional information, operational parameter information (such asdescribed elsewhere herein), and the like; (b) a training data set basedupon which an artificial intelligence system was trained in connectionwith the design or manufacturing of the item; (c) a sensor data set,such as containing time series sensor data (such as imaging data fromvarious imaging systems) indicating exact conditions of manufacturing ofthe item, such as linking a series of images of layers of the item as itwas generated with data indicating, in case with respect to the item,the environment in which it was manufactured, the equipment or toolsused, the materials used, and/or the like; temperatures, pressures,fluid flow rates, heat flux data, volume data, topological data,radiation data (e.g., intensity of lasers, visible light, infraredlight, UV, x-rays, magnetic fields, electrical fields and the like),chemical information (e.g., presence of reactants, catalysts, and thelike), biological data (e.g., presence and states of biomaterials,pathogens, and other factors), and others; (d) a testing data set, suchas indicating outcomes of testing before, during or after manufacturing,such as equipment testing, material testing, stress testing, visualinspection (including by machine vision), strain testing, torsiontesting, load testing, impact testing, operational testing, and thelike; (e) manufacturing information relating to similar items, such asoutcomes of manufacturing, usage, or the like; and others. Inembodiments, the additive manufacturing unit 10102 may automaticallycreate the digital twin upon receiving an instruction to manufacturingan item and subsequently enrich and/or modify the digital twin duringmanufacturing and/or after manufacturing. In embodiments, the additivemanufacturing unit 10102 may automatically embed the above-referenceddata for the digital twin of the item in or on the item (such as bywriting to a data structure that is embedded in or disposed on the item,such as chip), on a tag for the item, on a container or package, or thelike.

FIG. 116 is a block diagram illustrating the information flow in theautonomous additive manufacturing platform 10110 for optimization ofdifferent operational parameters of the additive manufacture processaccording to some embodiments of the present disclosure. In embodiments,the parameters may be associated with a 3D printed part, a 3D printedproduct, a 3D printing process, or a 3D printing machine. Some examplesfor parameters include: extrusion temperature, rate of materialdeposition, tool path, voltage settings of heating apparatus, exposurepattern, layer height, printing surface temperature, layerheight/thickness, build speed, build material flow rate, partorientation, air gap, shape and volume information for holes, spaces,voids, lumens, gaps, conduits and the like, support structure settings,ambient conditions including temperature, humidity and pressure, rawmaterial conditions including temperature and viscosity, part conditionsincluding temperature, stress concentrations including compressive,tensile, shear, bending and torsional stresses and the like. Again, theparameters are typically specific to a given additive manufacturingtechnique, material, geometry and application, or particular hybrid orcombination thereof

Referring to FIG. 116 , at 10300, input data for the printing of aproduct is received at the autonomous additive manufacturing platform10110. The input data may be received at a user interface of platform10110 and can include details like 3D printing technique, geometry andkey features of the product, and printing material etc. In embodiments,the input data may just include the required properties (like strength,stiffness, yield, elasticity, elongation, electrical conductivity,thermal conductivity etc.) or areas of application (aerospace, dental,automotive, jewelry etc,) of the product, and the platform 10110 maydetermine details like 3D printing technique or material to be used forprinting. This may occur automatically (such as by artificialintelligence), or with human interaction and/or supervision, such aswhere a set of recommended details are suggested by AI and confirmedand/or modified by a human user.

At 10302, an instruction set for additive manufacturing, such as aprofile, such as a 3D print profile, is determined based on the inputreceived at 10300 as well as simulation received from the machinelearning system 10210 and the digital twin system 10214. The profileincludes parameters for additive manufacturing of the product, such asusing the 3D printer.

At 10304, sensor data (including but not limited to ambient, product ormaterial temperatures; compressive, shear, tensile, bending andtorsional stresses; oxygen, carbon dioxide level, and ozone levels;humidity; vibration; sound signature and visual indicators) from theadditive manufacturing (e.g., 3D printing) process is collected. Thedata collection and management system 10202 helps collect the sensordata through an array of sensors and other data collecting technologieslike IoT devices, machine vision systems and the like. The collecteddata may be analyzed at the edge devices or sent to one or more datapools within the data storage system 10204 such as for later consumptionby local or remote intelligence. The use of cloud-connectable edgedevices, such as within computing infrastructure that is proximal to theadditive manufacturing unit(s) 10102 (such as in a local area network ofa building, campus, or other premises where the additive manufacturingunit(s) 10102 are located and/or in a connected vehicle that transportsthe additive manufacturing unit(s) 10102) and/or that is integrated withor into the additive manufacturing unit 10102, such as where theadditive manufacturing unit 10102 has onboard edge computational and/orconnectivity resources, such as 5G (or other cellular), Wifi, Bluetooth,fixed networking resources, or the like, offers opportunities to providerapid, real time or near real time processing responsiveness whilebenefiting from the expansive computing and data storage capabilitiesprovided by highly scalable cloud computing resources, such as serversand the like.

In embodiments, data may also be stored in a blockchain, such as onewhere storage is distributed across multiple manufacturing nodes as wellas other data storage devices or systems. In embodiments, this may takethe form of a distributed ledger that may capture transactions, events,or the like, such as financial events involving additive manufacturing,smart contract-related events, operational events (such as scheduling orcompletion of jobs), and others. The data may also be multiplexed orotherwise condensed using sensor fusion and relayed over a network andfed into the machine learning system employing one or more machinelearning models.

At 10306, the parameters may be dynamically adjusted as needed based onthe analysis of sensor data. As the 3D printing is complete, the datarelated to the outcome of the 3D printing process is collected at 10308.The outcome data may be collected through a user interface wherein auser provides information regarding the success or failure of the 3Dprint. The data is then provided as feedback to the machine learningsystem 10210 which uses the feedback to train or improve the initialmachine learning model (such as improvements by adjusting weights,rules, parameters, or the like, based on the feedback). In embodiments,the feedback is utilized to analyze trends over multiple 3D printsperformed by one or more users across multiple additive manufacturingunits 10102 and manufacturing nodes 10100.

In embodiments, the autonomous additive manufacturing platform 10110provides optimization and process control across the entire lifecycle ofmanufacturing using machine learning, from product conception and designthrough manufacturing and distribution to service and maintenance.

In embodiments, the autonomous additive manufacturing platform 10110provides for generative design and topology optimization to determine atleast one product design suitable for fabrication.

In embodiments, the autonomous additive manufacturing platform 10110provides for optimization of a build preparation process.

In embodiments, the autonomous additive manufacturing platform 10110optimizes part orientation process for superior production results.

In embodiments, the autonomous additive manufacturing platform 10110automatically determines and recommends support structures to minimizematerial costs, print time, post processing, and risk of damage to the3D printed part (on support removal).

In embodiments, the autonomous additive manufacturing platform 10110provides for optimizing toolpath generation. For example, in a 3Dprinter, a toolpath may comprise the trajectory of the nozzle and/orprint head. In embodiments, toolpath generation enables a manufacturingprocess to fill the boundary and interior areas of each sliced layer.Various types of toolpath strategies and algorithms, such as zigzag,contour, spiral and partition patterns, are possible with considerationson the build time, cost, geometrical quality, warpage, shrinkage,strength and stiffness of a manufacturing model. In embodiments, anartificial intelligence system may be trained on outcomes, such asdescribed above, to provide a recommended toolpath and/or to entirelyautomate toolpath generation.

In embodiments, the autonomous additive manufacturing platform 10110provides for optimized dynamic 2D, 2.5D and 3D nesting to maximize thenumber of printed parts while minimizing the raw material waste. Inembodiments, nesting is optimized such that the nesting algorithmevaluates individual part priority to ensure high priority parts arehandled accordingly, such as with scheduling priority, priority inquality, priority in ease-of-use, priority of positioning, or the like.In embodiments, nesting is optimized such that the nesting algorithmminimizes the travel time for the cutting tool. In embodiments, nestingis optimized such that the nesting algorithm integrates with supportstructure optimization.

In embodiments, the autonomous additive manufacturing platform 10110provides for optimization of post processing processes.

In embodiments, the autonomous additive manufacturing platform 10110provides for an automated powder removal system utilizing a digital twinwherein the digital twin calculates the optimal movement of the powderremoval system while de-powdering.

In embodiments, the autonomous additive manufacturing platform 10110provides for an automated, hands-free support structure removal.

In embodiments, the autonomous additive manufacturing platform 10110provides for automated surface finishing.

In embodiments, the autonomous additive manufacturing platform 10110provides for automated part metrology for use with integrated qualityand process control systems.

In embodiments, manufacturing methods described herein may use materialadditives during processing that impart various characteristics infinished parts. Examples in plastic injection molding include glassfiber for added strength, and electrically conductive and shieldingfibers for tailored electrical properties. For some applications,orientation of added fibers or other materials may affect theperformance of finished parts. For example, in a glass fiberreinforcement application, long fiber orientation may dictate minimumand maximum deformation orientations under stress. Fiber orientationduring manufacturing may be only partially controlled through molddesign, injection nozzle location and pressure, and other processcontrols.

3D printed parts may also be manufactured using material additives;however, most 3D printing methods can only produce materials withlimited ability to optimize additive characteristics such as fiberorientation to help optimize finished part performance. For example, 3Dprinters may use nozzles that extrude various plastic materials, butinherent flow characteristics of a fixed nozzle, and limitations of the3D printing process in general, limit options for finished part materialengineering. Such use of 3D printing nozzles offer the ability tocontrol orientation of additive materials as they are laid down for partproduction. This development provides the opportunity to finely tailormaterial performance, for example, localized orientations for structuralenhancement, or homogeneous random orientation for electrical shieldingperformance. In examples, this capability may be provided by a 3Dprinting nozzle that uses actuated flexible elements to change the shapeof the nozzle during material application, resulting in predictablefiber orientations. This may be used in conjunction with other printingprocess parameters such as nozzle orientation, flow rate and pressure,and the like to further refine material characteristics. Use caseexamples include, but are not limited to: one or more engineeringcharacteristics that may vary across a single part to provide targetedperformance, for example varying stiffness; optimized use of materialsbased on enhanced process control, for example using less material toproduce a part with the same functional performance, and providingcontrol of multiple additives to impart combined capabilities, forexample orientation of structural long fibers for structuralperformance, combined with randomized conductive additives for aspecified electrical performance.

In embodiments, of the present disclosure, including ones involvingartificial intelligence, machine learning, automation (including roboticprocess automation, remote control, autonomous operation, automatedconfiguration, and the like), expert systems, self-organization,adaptive intelligent systems for prediction, classification,optimization, and the like, may benefit from the use of a neuralnetwork, such as a neural network trained for pattern recognition, forclassification of one or more parameters, characteristics, or phenomena,for support of autonomous control, and other purposes.

Neural networks (or artificial neural networks) are a family ofstatistical learning models inspired by biological neural networks andare used to estimate or approximate functions that may depend on a largenumber of inputs and are generally unknown. Neural networks represent asystem of interconnected “neurons” which send messages to each other.The connections have numeric weights that can be tuned based onexperience, making neural nets adaptive to inputs and capable oflearning.

References to artificial intelligence, neural networks or neural netthroughout this disclosure should be understood to encompass a widerange of different types of machine learning systems, neural networks,such as feed forward neural networks, convolutional neural networks(CNN), recurrent neural networks (RNN), long short-term memory (LSTM)neural networks, gated recurrent unit (GRU) neural networks,self-organizing map (SOM) neural networks (e.g., Kohonen self-organizingneural networks), autoencoder (AE) neural networks, encoder-decoderneural networks, modular neural networks, or variations, hybrids orcombinations of the foregoing, or combinations with reinforcementlearning (RL) systems or other expert systems, such as rule-basedsystems, and model-based systems (including ones based on physicalmodels, statistical models, flow-based models, biological models,biomimetic models and the like).

The foregoing neural networks may have a variety of nodes or neurons,which may perform a variety of functions on inputs, such as inputsreceived from sensors or other data sources, including other nodes topredict one or more outputs. Functions may involve weights, features,feature vectors, and the like. Neurons may include perceptrons, neuronsthat mimic biological functions (such as the human senses of touch,vision, taste, hearing, and smell), and the like. Neural networks canemploy multiple layers of operations including one or more hidden layerssituated between an input layer and an output layer. The output of eachlayer can be used as input to another layer, e.g., the next hidden layeror the output layer. The output of a particular neuron can be a weightedsum of the inputs to the neuron, adjusted with a bias and multiplied byan activation function, e.g., a rectified linear unit (ReLU) or asigmoid function.

In many embodiments, an expert system or neural network may be trained,such as by a human operator or supervisor, or based on a data set,model, or the like. Training a neural network can involve providinginputs to the untrained neural network to generate predicted outputs,comparing the predicted outputs to expected outputs, and updating thealgorithm's weights and biases to account for the difference between thepredicted outputs and the expected outputs. Specifically, a costfunction can be used to calculate a difference between the predictedoutputs and the expected outputs. By computing the derivative of thecost function with respect to the weights and biases of the network, theweights and biases can be iteratively adjusted over multiple cycles tominimize the cost function. Training may be complete when the predictedoutputs satisfy a convergence condition, e.g., a small magnitude ofcalculated cost as determined by the cost function.

Training may include presenting the neural network with one or moretraining data sets that represent values (including the many typesdescribed throughout this disclosure), as well as one or more indicatorsof an outcome, such as an outcome of a process, an outcome of acalculation, an outcome of an event, an outcome of an activity, or thelike. Training may include training in optimization, such as training aneural network to optimize one or more systems based on one or moreoptimization approaches, such as Bayesian approaches, parametric Bayesclassifier approaches, k-nearest-neighbor classifier approaches,iterative approaches, interpolation approaches, Pareto optimizationapproaches, algorithmic approaches, and the like. Feedback may beprovided in a process of variation and selection, such as with a geneticalgorithm that evolves one or more solutions based on feedback through aseries of rounds.

In embodiments, a plurality of neural networks may be deployed in acloud platform that receives data streams and other inputs collected(such as by mobile data collectors) in one or more environments andtransmitted to the cloud platform over one or more networks, includingusing network coding to provide efficient transmission. In the cloudplatform, optionally using massively parallel computational capability,a plurality of different neural networks of various types (includingmodular forms, structure-adaptive forms, hybrids, and the like) may beused to undertake prediction, classification, control functions, andprovide other outputs as described in connection with expert systemsdisclosed throughout this disclosure. The different neural networks maybe structured to compete with each other (optionally including useevolutionary algorithms, genetic algorithms, or the like), such that anappropriate type of neural network, with appropriate input sets,weights, node types and functions, and the like, may be selected, suchas by an expert system, for a specific task involved in a given context,workflow, environment process system, or the like.

In embodiments, methods and systems described herein that involve anexpert system or self-organization capability may use a feed forwardneural network, which moves information in one direction, such as from adata input, like a source of data about an individual, through a seriesof neurons or nodes, to an output. Data may move from the input nodes tothe output nodes, optionally passing through one or more hidden nodes,without loops. In embodiments, feed forward neural networks may beconstructed with various types of units, such as binary McCulloch-Pittsneurons, the simplest of which is a perceptron.

In embodiments, artificial intelligence and machine learning systems inthe data processing system of the autonomous additive manufacturingplatform 10110 may enable automatic classification and clustering of 3Dprinted parts and products. In embodiments, artificial intelligence andmachine learning systems in the data processing system of the autonomousadditive manufacturing platform 10110 may enable automaticclassification and clustering of malicious defects in the additivemanufacturing process.

The automated part and defect classification methods and systems of thepresent disclosure may be implemented using image sensors and/or machinevision systems. The machine vision systems may monitor the additivemanufacturing process in real time, such as by capturing and analyzingimages of the part or other item being printed. Automated imageprocessing of the captured images may then be used to monitor any of avariety of part properties, e.g., dimensions (overall dimensions, ordimensions of specific features), feature angles, feature areas, surfacefinish (e.g., degree of light reflectivity, number of pits and/orscratches per unit area), and the like. The machine vision systems alsotrack the process to detect any defects or errors in the printed part inreal time while successive layers of materials are being deposited bythe 3D printer.

Defects may be identified, e.g., by removing noise from the inspectiondata and subtracting a reference data set (e.g., a reference image of adefect-free part in the case that machine vision tools are beingutilized for inspection), and classified using an unsupervised machinelearning algorithm such as cluster analysis or an artificial neuralnetwork, to classify individual objects as either meeting or failing tomeet a specified set of decision criteria (e.g., a decision boundary) inthe feature space in which defects are being monitored. For example, apartially printed part may be compared with a render of the partial partand in case the partial part differs beyond a selected threshold fromthe render, the part may be classified as defective.

In embodiments, in-process the defect classification data may be used bythe machine learning algorithm to determine a set or sequence of processcontrol parameter adjustments that will implement a corrective action,e.g., to adjust a layer dimension or thickness, so as to correct adefect when first detected. In some embodiments, in-process automateddefect classification may be used by the machine learning algorithm tosend a warning or error signal to an operator, or optionally, toautomatically abort the deposition process.

In embodiments, the machine vision system uses a variable focus liquidlens-based camera for image capture and defect detection. Inembodiments, the machine vision system uses infrared or visiblewavelength cameras.

In embodiments, the data processing system is implemented as theintelligence layer 140 that uses a neural network to provide real-time,adaptive control of an additive manufacturing process including partdefect classification and feedback.

In some embodiments, a neural network model may be used directly todetermine adjustments to process control parameters using training orlearning of a neural network model. Initially, the model is allowed tochoose randomly from a range of values for each input process controlparameter or action. If the sequence of process control parameteradjustments or actions leads to a flaw or defect, it is scored asleading to an undesirable (or negative) outcome. Repetition of theprocess using different sets of randomly chosen values for each processcontrol parameter or action leads to reinforcement of those sequencesthat least to desirable (or positive) outcomes. Ultimately, the neuralnetwork model “learns” what adjustments to make to a set or sequence ofdeposition process control parameters or actions in order to achieve thetarget outcome, i.e., a defect-free printed part.

In embodiments, methods and systems described herein may use aconvolutional neural network (referred to in some cases as a CNN, aConvNet, a shift invariant neural network, or a space invariant neuralnetwork), wherein the units are connected in a pattern similar to thevisual cortex of the human brain. For example, the CNN may provideautomatic classification and clustering of parts and defects in anadditive manufacturing process.

In embodiments, one or more models building on the basic framework ofconvolutional neural networks may be employed. For example, an objectdetection model may be used that extends the functionality of CNN basedimage classification models by not only classifying parts or defects butalso determining their locations in an image in terms of bounding boxes.Similarly, Region-based CNN (R-CNN) models may be used to extractregions of interest (ROI), where each ROI is a rectangle that mayrepresent the boundary of a part in image.

In embodiments, Capsule Networks may be employed to use fewer labeledtraining examples to achieve similar classification performance of CNNs.

In embodiments, transformer-based, encoder-decoder architectures usingattention mechanisms may be used in conjunction with or in place ofconvolutional neural networks.

FIG. 117 is a schematic view illustrating a system for learning on datafrom the platform 10110 to train the artificial learning system to usedigital twins for classification, predictions and decision-makingaccording to some embodiments of the present disclosure.

Referring to FIG. 117 , the digital twin system 10214 in the autonomousadditive manufacturing platform 10110 may include product twins 10215,part twins 10504, printer twin 10506, user twin 10508, manufacturingnode twin 10510, packager twin 10512 and the like, that allow formodeling, simulation, prediction, decision-making, and classification.The digital twin system 10214 may be populated with relevant data, forexample the product twins 10215 may be populated with data related tocorresponding product including dimension data, material data, featuredata, thermal data, price data, and the like

In embodiments, a digital twin may be generated from other digitaltwins. For example, the product twin 10215 may be generated using one ormore part twins 10504. In another example, the part twins 10504 may begenerated using the product twins 10215. In embodiments, a digital twinmay be embedded in another digital twin. For example, the part digitaltwin 10504 may be embedded in the product digital twin 10215 which maybe embedded in the manufacturing node digital twin 10510.

In embodiments, a simulation management system 10514 may set up,provision, configure, and otherwise manage interactions and simulationsbetween and among digital twins 10214.

In embodiments, the artificial intelligent system 10212 is configured toexecute simulations in a simulation management system 10514 using thepart twins 10502 and/or other digital twins available to the digitaltwin system 10214. For example, the artificial intelligent system 10212may adjust one or more features of the printer twin 10506 as a set ofpart twins 10504 are printed by the 3D printer. In embodiments, theartificial intelligent system 10212 may, for each set of features,execute a simulation based on the set of features and may collect thesimulation outcome data resulting from the simulation. For example, inexecuting a simulation on the set of part twins 10504 being manufacturedin the printer twin 10506, the artificial intelligent system 10212 canvary the properties of the printer twin 10506 and can executesimulations that generate outcomes. During the simulation, theartificial intelligent system 10212 may vary the ambient temperature,pressure, humidity, lighting, and/or any other properties of the printertwin 10506. In this example, an outcome can be a condition of the parttwin 10504 after being subjected to a high temperature. The outcomesfrom simulations can be used to train the machine learning models 10213.In embodiments, the machine learning system 10210 may receive trainingdata, outcome data, simulation data, and/or any other data from otherdata sources 10114. In embodiments, the machine learning system 10210may train/reinforce the machine learning models 10213 using the receiveddata to improve the models.

In embodiments, the machine-learning system 10210 trains one or moremodels that are utilized by the artificial intelligence system 10212 tomake classifications, predictions, recommendations, and/or to generateor facilitate decisions or instructions relating to the product and thepart, such as decisions or instructions governing design, configuration,material selection, shape selection, manufacturing type, job schedulingand many others.

In example embodiments, the artificial intelligence system 10212 trainsa part failure prediction model. A failure prediction model may be amodel that receives part related data and outputs one or morepredictions or answers regarding the probability of part failure. Thetraining data can be gathered from multiple sources including partspecifications, environmental data, sensor data, machine vision data andoutcome data. Some examples of questions that the prediction model mayanswer are: when will the machine fail, what type of failure it will be,what is the probability that a failure will occur within the next Xhours, what is the remaining useful life of the part, and the like. Theartificial intelligence system 10212 may train one or more predictionmodels to answer different questions. For example, a classificationmodel may be trained to predict failure within a given time window,while a regression model may be trained to predict the remaining usefullife of the machine. In embodiments, training may be done based onfeedback received by the system, which is also referred to as“reinforcement learning.” The artificial intelligence system 10212 mayreceive a set of circumstances that led to a prediction (e.g.,attributes of part, attributes of a model, and the like) and an outcomerelated to the part and may update the model according to the feedback.

In embodiments, the artificial intelligence system 10212 may use aclustering algorithm to identify the failure pattern hidden in thefailure data to train a model for detecting uncharacteristic oranomalous behavior. The failure data across multiple parts and theirhistorical records may be clustered to understand how different patternscorrelate to certain wear-down behavior. For example, if the failurehappens early in the print, the failure may be due to uneven printsurface. If the failure occurs later on in the print, it is likely thatthe part became detached from the printing surface and the cause offailure is poor bed adhesion and/or warping. All of the informationgathered can be used as feedback for the model. Over time, variousfailure modes will become associated with corresponding parameters. Forexample, poor bed adhesion is likely caused by incorrect temperaturesettings or printing orientation. Any failure to meet dimensionaltolerances is likely caused by incorrect acceleration, speed, or layerheight. The machine-learning system 10212 can determine the degree ofcorrelation between each input and each failure mode.

In embodiments, the artificial intelligence system 10212 may beconfigured to monitor cutting tools, filters and machine lasers toinitiate maintenance or replacement as needed including platform-widemaintenance management, and as part of computerized maintenancemanagement systems (MMS). In embodiments, additive manufacturingentities of a value chain network may be prepared, configured and/ordeployed to support replacement of parts. For example, in connectionwith a service visit to a home or business, an additive manufacturingunit may be designated to support the service visit, such as a mobileadditive manufacturing unit and/or a unit located in sufficiently closeproximity to the service visit to facilitate rapid delivery of itemsproduced by the additive manufacturing unit. Based on the nature of theservice visit (e.g., the type of equipment to be serviced, the nature ofcomponent parts and materials in the equipment, identified problems, andthe like), the additive manufacturing unit may be equipped withappropriate materials, such as a combination of metal printing materialsand other printing materials, that are suitable to print a range ofpossible replacement parts, specialized tools, or other elements tosupport the service visit. In embodiments, the platform may take inputsfrom or related to the service visit, such as inputs indicating the itembeing serviced (e.g., technical specifications, CAD designs, and thelike); inputs indicating diagnosed issues (such as a need to replace anentire sub-assembly, a need to repair a crack or other damage, or thelike); and inputs captured by cameras, microphones, data collectors,sensors, and other information sources associated with the servicevisit. For example, a service technician may capture a set of photosthat show a damaged part. In embodiments, the platform may process theinputs, such as using an artificial intelligence system (such as arobotic process automation system trained on a training set of expertservice visit data), to determine a recommended action, which inembodiments may involve replacement of a part and/or repair of a part.The platform may, in some such embodiments, automatically determine(such as using an artificial intelligence system, such as roboticprocess automation trained on an expert data set) whether a replacementpart is readily available and/or whether an additive manufacturingsystem should produce the replacement part, such as to reduce delay, tosave costs, or the like. Similarly, the platform may, in someembodiments, using similar systems, automatically determine that anelement should be additively manufactured to facilitate repair, such aswhere a complementary component may be generated to replace a worn orabsent element. In embodiments, automatic determination may occur usinga machine vision system that captures a set of photo images from theservice visit, compares them to reference designs for applicable partsand produces an instruction set for additively manufacturing acomplementary element that can be added (such as by being adhered with aspecified adhesive) to a defective element in order to render the partin compliance with the reference design. In any such embodiment thatrecommend or configure instructions for additive manufacturing, theplatform may discover available units, configure instructions, andinitiate additive manufacturing, and provide updates to the servicetechnician, such as updates as to when an element will be ready to use.In embodiments, the platform, such as through a trained AI agent, mayautomatically configure and schedule a set of jobs across a set ofadditive manufacturing units with awareness of the status of otherrelevant entities involved in service and other workflows, such as theoverall planned duration of a service job (e.g., to allowde-prioritization of additive manufacturing jobs that will produceoutputs that won't be used immediately), what other work is being done(e.g., to allow for appropriate sequencing of additive manufacturingoutputs that align with overall workflows), the priority of the servicejob (e.g., whether it relates to a mission critical item of operatingequipment, versus a non-critical accessory item), the cost of downtime,or other factors. In embodiment, optimization of workflows across a setof additive manufacturing entities may occur by having an artificialintelligence system undertake a set of simulations, such as simulationsinvolving alternative scheduling sequences, design configurations,alternative output types, and the like. In embodiments, simulations mayinclude sequences involving additive manufacturing and othermanufacturing entities (such as subtractive manufacturing entities thatcut, drill, or the like and/or finishing entities that polish, cure, orthe like), including handoffs between sets of different manufacturingentity types, such as where handoffs are handled by robotic handlingsystems. In embodiments, a set of digital twins may represent attributesand capabilities of the various manufacturing systems, various handlingsystems (robotic systems, arms, conveyors, and the like, as well ashuman workforce) and/or the surrounding environment (such as a vehicle,a manufacturing facility, a campus, or even a larger scale entity, suchas a city).

In embodiments, the artificial intelligence system 10212 may beconfigured to manage the real time dynamics affecting inventory levelsfor smart inventory and materials management. This may include, forexample, forecasting inventory levels based on a set of demand factorsand/or supply factors of various types described herein and configuringschedules for additive manufacturing units 10102 to produce items forlocations where shortages are anticipated.

In embodiments, the artificial intelligence system 10212 may beconfigured to build, maintain, and provide a library of parts withpreconfigured parameters, that may be searchable by materials,properties, part type, part class, industry, compliance, etc. This mayinclude, for example, a set of search algorithms that discover parts byreferencing published materials, including website materials, productspecifications, or the like; a set of algorithms that query APIs orother interfaces of parts providers, such as to query databases forparts information; and/or a set of data collection systems that captureimages, sensor data, test data, or the like of or about parts.

In embodiments, the artificial intelligence system 10212 may beconfigured to analyze usage patterns associated with one or more usersand learning user preferences with respect to outputs, timing,materials, colors, shapes, orientations, and/or print strategies. Forexample, the system 10212 may develop a profile, such as by the additivemanufacturing unit 10102, by location, by user, by organization, byrole, or the like, that indicates what materials were used formanufacturing, what processes were used for manufacturing, what shapeswere produced, what finishing steps were undertaken, what colors wereused, what functions were enabled, and the like. The profile may be usedto determine, infer, or suggest preferences of users, organizations, orthe like. For example, an organization's preferred brand colors may berecognized, such that conforming materials and coatings are recommendedand/or preconfigured in development of additive manufacturing steps.

In embodiments, the artificial intelligence system 10212 may beconfigured to perform real time calibration for one or more 3D printers.This may include training on a training data set of calibrationinteractions of expert users. Calibration may be job-specific, such asby training the artificial intelligence system 10212 to calibrate theadditive manufacturing unit 10102 to operate with a specific material,which may include material from a specific bin or lot of the samegeneral type of material.

In embodiments, the artificial intelligence system 10212 may beconfigured to minimize the material waste production during the additivemanufacturing process. This may include configuring production tominimize material that needs to be removed in finishing steps,configuring production to produce outputs where unused material iseasily removed for reuse, and/or configuring production to favorreusable/recyclable materials.

In embodiments, the artificial intelligence system 10212 may beconfigured to detect cyber security risks and threats to the platform10110.

In embodiments, the artificial intelligence system 10212 may beconfigured to assess regulatory compliance. For example, in embodimentsthe artificial intelligence system 10212 may be configured to search alibrary or other source of approved or certified product designs, suchas ones that are UL or CE certified, FDA approved, OSHA-approved, or thelike and compare a design configuration to the same to confirm that anoutput of additive manufacturing will result in a compliant/approvedform of product. In embodiments, the artificial intelligence system10212 may work with a digital twin system, a simulation system, or thelike to simulate performance of a resulting output and may compare thesimulated performance to regulatory or other requirements, such as onesapplying to the ability to withstand forces, chemical effects,biological effects, radiation, or the like. For example, where a productcomponent, such as a housing, is intended to provide shielding fromradiation, the artificial intelligence system 10212 may operate on orwithin a digital twin that includes a radiation propagation physicsmodel to automatically assess whether product materials, thicknesses,and shapes will provide shielding sufficient to meet regulatory and/ordesign requirements.

In embodiments, the artificial intelligence system 10212 may beconfigured to optimize power consumption for the platform 10110. Thismay include training the artificial intelligence system 10212 on atraining set of operational data that includes (a) measuring powerconsumed by various available activities; (b) training the artificialintelligence system 10212 to undertake scheduling of additivemanufacturing jobs according to a predictive model of energy pricing;and/or (c) having the artificial intelligence system undertake a largebody of simulations to select a preferred sequence of operations thatproduces a favorable power consumption pattern.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins for predicting part shrinkage or expansion.This may include having the artificial intelligence system 10212 use aset of physical models that include thermal coefficients of expansionfor elements, alloys, compounds, mixtures and/or combinations,including, in embodiments, graded layers of material where there is nota clear boundary between materials. In embodiments, the artificialintelligence system 10212 may be trained based on observed shrinkingand/or expansion during manufacturing and/or use.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins for predicting part warpage. This may includehaving the artificial intelligence system 10212 use a set of physicalmodels that include thermal coefficients of expansion for elements,alloys, compounds, mixtures and/or combinations, including, inembodiments, graded layers of material where there is not a clearboundary between materials. In embodiments, the artificial intelligencesystem 10212 may be trained based on observed warpage duringmanufacturing and/or use.

In embodiments, the models trained by the machine learning system 10210may be utilized by the artificial intelligence system 10212 to executesimulations on part twins for calculating necessary changes to the 3Dprinted process to compensate for part shrinkage, expansion and/orwarpage.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins for testing the compatibility of additivelymanufactured parts. In embodiments, the compatibility may be tested withone or more other parts in an assembly. In embodiments, thecompatibility may be tested with an operating environment. Inembodiments, the compatibility may be tested with a 3D printer.Compatibility may include shape compatibility (e.g., key-in-lock;housing-around-interior; peg-in-hole; male-with-female,support-with-supported, or other types of interface/interconnectcompatibility); environmental compatibility (e.g., compatibility ofmaterials with anticipated environment of use, such as chemical factors,physical factors, radiation factors, biological factors, temperatures,pressures and the like); functional compatibility (e.g., ability towithstand loads, stresses, torsion, or the like) and others.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins for predicting deformations or failure in anadditively manufactured item.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins for optimizing the build process to minimizethe occurrence of deformations.

In embodiments, the models trained by the machine learning system 10210may be utilized by the artificial intelligence system 10212 to executesimulations on product twins for predicting the price of a product. Inembodiments, prediction of a price may include: (a) prediction based onmarket prices of similar items (and/or forecasts of such prices); (b)prediction based on predicted demand; (c) prediction based on committeddemand; (d) prediction based on smart contract terms and conditions;and/or (e) prediction based on cost, including materials, energy costs,shipping, and labor, among others (which may include a range ofprofit/markup amounts to arrive at a price from a base cost). Inembodiments, price prediction may include wholesale pricing, retailpricing, volume pricing, location-based pricing, and the like.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins and printer twins forgenerating additive manufacturing quotes.

In embodiments, the models trained by the machine learning system 10210may be utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins and printer twins forgenerating recommendations related to printing to a user of theplatform. In embodiments, the recommendations may relate to a choice ofa material for printing. In embodiments, the recommendations may relateto a choice of an additive manufacturing technique. In embodiments,recommendations may relate to timing of manufacturing.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins and printer twins forpredicting delivery times for additive manufacturing jobs. Simulationsmay include ones that vary at a level of priority to determine apredicted delivery time under different priority levels (such as toindicate tradeoffs between latency and price/cost).

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins, printer twins, manufacturingnode twins or others for predicting cost over-runs in the manufacturingprocess.

In embodiments, the models trained by machine learning system 10210 maybe utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins, printer twins andmanufacturing node twins for optimizing the production sequencing ofparts based on quoted price, delivery, sale margin, order size, orsimilar characteristics. In embodiments, optimization may includeoptimization based on public data, such as market data, website data,manufacturer-provided data (such as by APIs) and/or terms and conditionsof a set of smart contracts that relate to such characteristics.

In embodiments, the models trained by the machine learning system 10210may be utilized by the artificial intelligence system 10212 to executesimulations on part twins, product twins and printer twins foroptimizing the cycle time for manufacturing. In embodiments, theoptimizing of cycle time includes time for post-processing (which canvary dramatically per part specifications and additive manufacturingtechnology).

In embodiments, an instruction set for additive manufacturing may beautomatically generated from a text description, such as using a blendof natural language-based artificial intelligence and other artificialintelligence for handling and/or generating images and/or spatialrepresentations, such as using the DALL-E language model from OpenAI™ orother transformer language model (a combination of text-based andimage-based models) further combined with a model for transforming animage into a 3D model and/or a model for transforming an image or 3Dmodel into an additive manufacturing instruction set. The hybrid,transformer artificial intelligence system may, for example, be trainedto generate a set of parameters that represent a set of semantic objects(such as a pair of glasses and a cat), generate an output design (suchas glasses that have catlike attributes, such as whiskers or cats-eyelenses), and convert the output design into an additive manufacturinginstruction set. In such embodiments, a user may, for example, enter atext string for a desired output and be provided with a range of 3Dmodels representing options. The user may select the preferred optionand initiate an additive manufacturing job to product the item. Inembodiments, the platform may track interests, attributes, searchresults, profiles, news topics, or other factors to generate a set ofinput text strings to produce a set of objects that are recommended foradditive manufacturing for a user. In embodiments, recommendations arebased on similarity to other users, such as based on clusteringtechniques. In embodiments, recommendations are based on collaborativefiltering.

In embodiments, the digital twin system 10214 are configured tocommunicate with a user via multiple communication channels such asspeech, text, gestures, and the like. For example, the digital twin mayreceive queries from a user about the distributed manufacturing networkentities, generate responses for the queries and communicate suchresponses to the user. Additionally, digital twins may communicate withone another to learn from and identify similar operating patterns andissues in other distributed manufacturing network entities, as well assteps taken to resolve those issues. For example, the digital twins oftwo manufacturing nodes or those of a part, a printer and amanufacturing node may communicate with one another for resolving oranswering a customer request.

FIG. 118 is a schematic illustrating an example implementation of anautonomous additive manufacturing platform including various componentsalong with other entities of a distributed manufacturing networkaccording to some embodiments of the present disclosure.

The autonomous additive manufacturing platform 10110 may collect datafrom one or more entities including users, programs and the data sources10114. A data acquisition system 10602 in user interface 10112 mayinclude a set of interfaces like a chat interface 10604, a smart voiceinterface 10606 and a file upload interface 10608 to collect data fromone or more users of the platform. Additionally, one or more sensors10610 including camera and machine vision system, acoustic/sound sensors(e.g., with microphones, including optionally multiple microphones in anarray), power and energy sensor, mass sensor, location sensor,temperature sensor, humidity sensor, pressure sensor, viscosity sensor,flow sensor, chemical/gas sensor, strain gauge, thermal imaging,hyperspectral imaging, sound sensor, air quality sensor and the like mayprovide data to the platform 10110. The data sources 10114 may alsoinclude programs, the feedback sources 10612 providing outcome data fromthe machine learning system 10210 and a data library 10614.

In embodiments, a data visualization 10615 in the user interface 10112may provide a set of dashboards, interfaces and integrations for a userof the platform 10110 to visualize information related to thedistributed manufacturing network 10130 or one or more entities in thenetwork 10130. For example, a dashboard may provide visualizationsincluding information related to digital threads for distributedmanufacturing network entities like a 3D printed part or a product.Another dashboard may provide visualizations including information aboutreal time visibility of status of a manufacturing order. An alternatedashboard may provide visualizations including information related tobatch traceability to identify parts from the same batch. A dashboardmay provide visualization of demand factors, including predicted demand,inventory levels and the like. A search interface may be provided toresolve queries from one or more users based on part, machine,production date or location. In embodiments, a virtual reality (VR)system may be integrated with the data visualization 10615 and modellingsystem 10620, thereby enabling a user to build 3D models in VR. Inembodiments, the virtual reality system may be integrated with ascanning system 10617, such as allowing a user to build models thatconsist of scanned data (such as point clouds) and/or combinations ofmodel-based VR and scans (and/or other augmentations or overlays, suchas in augmented reality and/or mixed reality models). This may alsoinclude a wider set of user interactions for developing part designswithout in-depth expertise including using augmented reality (AR) andmixed reality (MR).

In embodiments, the user interface 10112 may include a single clickpre-processing process triggering pre-set configurations for partorientation, support determination, toolpath generation and/or nesting.

In embodiments, the user interface 10112 may include a single clickpost-processing process triggering pre-set configurations forde-powdering, support removal and surface finishing.

A user of the platform may also use the design and simulation system10116 to build CAD and STL files capturing the design of the part orproduct to be printed. A set of design tools 10616 and design libraries10618 may allow a user to build models in modelling system 10620 and runsimulations in simulation environment 10622. In embodiments, the designof the part or product may be captured in various file formats includingbut not limited to, IGES files, SolidWorks files, Catia files, ProEfiles, 3D Studio files, STEP files and Rhino files. In embodiments, thedesign may be captured in the form of digital images, such as in PNGfiles, JPEG files, GIF files and/or PDF files, as well as scanned dataformats, such as point clouds produced by laser scanning, and outputsfrom ultrasound, MRI, x-Ray, electron beam, radar, IR and other scanningsystems.

The data storage system 10204 may store data in a distributed ledger10624, a digital thread 10626 or the like, such as for maintaining arecord of event data 10628 and a state data 10630 for an entity or assetof the distributed manufacturing network 10130 over time, including apart or products or any other asset or entity described herein.

In embodiments, the digital thread 10626 constitutes information relatedto the complete lifecycle of an item produced by additive manufacturing,such as a part, from design, modeling, production, validation, use andmaintenance through disposal.

In embodiments, the digital thread 10626 constitutes information relatedto one or more additive manufacturing machines, or tools includingpost-processing tools such as CNC equipment, robotics support,product/part marking, metrology equipment and the like across multiplemanufacturing facilities/locations.

In embodiments, the digital thread 10626 constitutes information relatedto the complete lifecycle of a product from design, modeling,production, validation, use and maintenance through disposal, optionallyincluding aggregated, linked, or integrated information from multipleconstituents into a full product digital thread.

The data processing system 10206 processes the data collected by datacollection and the management system 10202 to optimize and adjust theprocess parameters in real time through the artificial intelligencesystem 10212 (including the machine learning system 10210), the digitaltwin system 10214 and the control system 10216 as described in detail inFIGS. 115, 116 and 118 or elsewhere herein or in the documentsincorporated herein by reference.

The manufacturing workflow management applications 10208 may manage thevarious workflows, events and applications related to production orprinting and value chain management. In embodiments, a matching system10632 may help with matching a set of customer orders with a set ofadditive manufacturing units 10102 or manufacturing nodes. Orders mayinclude firm orders, contingent orders (e.g., based on pricecontingency, timing contingency or other factors), aggregated orders,custom orders, volume orders, time-based orders, and others. Inembodiments, orders may be expressed in smart contracts, such asoperating on a set of blockchains. The matching may be based on factorslike additive manufacturing capabilities, locations of the customer andthe manufacturing nodes, available capacity at each node, materialavailability, pricing (including materials, energy, labor andopportunity costs of other available uses for capacity) and timelinerequirements. In embodiments, different parts of a product may bematched with different manufacturing nodes and the product may beassembled at one of the nodes, or elsewhere in a value chain network(such as while in transit, such as by a robotic assembly system locatedin a vehicle or shipping container), before being finally delivered tothe customer.

In embodiments, the additive manufacturing platform may be configured tomaintain an inventory of parts available to large airplane or sea-goingsystems in which multiple redundancies are mandated by custom and/orregulation. In embodiments, example systems include double, triple ormore redundancies over primary operation systems. In these examples,certain systems may benefit from ready-to-be made products filling infor the third, fourth, etc. redundancy when previously a full inventoryto adequately supply the entire third, fourth, etc. redundancy wasrequired. It will be appreciated in light of the disclosure that not allsystems will be applicable in that some critical systems may only permitsuch parts as further layers of redundancies to the already mandatedsupplies. While in flight, the desire to minimize weight and energyconsumption may limit the desire for the creation of certain parts, theability to generate parts on longer endurance flights to attend to theneeds of the cabin may be one motivation to provide some inflightfunctionality. For example, locking components that may fail midflight,such as latches, hinges, seat-belts, and the like, can be replaced ortemporarily locked closed to improve in-cabin safety. Components thatmay have come loose may also be shimmed or temporarily lodged in placeby a custom printed part to wedge or hold parts in place through theflight. Examples include holding avionic components in a dashboard,overhead, or other cockpit controls, holding hospitality items in thegalley, holding seats on seat rails, and the like.

In an example, the additive manufacturing platform can be used to createadditional inventory to outfit the airplane for items constructibleinflight that are required on the minimum equipment list to fly and havethose parts replaced before the airplane lands and returns to the gatefor service thus at least contributing to a repair that otherwise wouldnot require an early landing but may prevent the next dispatch of theairplane to its next desired use.

In sea-going embodiments, the additive manufacturing platform may beused to create additional inventory to outfit the sea going vessel withitems constructible during the voyage that are required on the mandatedminimum equipment list to embark (or the like) and have those partsreplaced before the vessel moors and reloads thus at least contributingto a repair that otherwise would not require a detour and coming ashoreearly but may prevent the next timely dispatch of the vessel to its nextdesired use.

In embodiments, the additive manufacturing platform may be configured tocoordinate with land-based additive manufacturing assets to coordinateconstruction of parts and coordinated portions of greater assemblies sodowntime in port or in the hanger can be minimized. In this example,entities providing just in time maintenance inventories can extend theirreach and depth by augmenting their one or more offerings orcoordinating their one or more offerings within port or in hangersystems that can be coordinating with one or more in-situ systems activeduring voyage and/or flight.

In embodiments, the matching system 10632 helps with matching anadditive manufacturing task with an engineer where the matching may bebased on factors like task complexity, engineer experience andexpertise. In embodiments, the matching system 10632 helps with matchingan additive manufacturing task with the location and/or availability ofa finishing worker where the matching may be based on factors like taskcomplexity, worker experience and expertise. In embodiments, thematching system 10632 helps with matching an additive manufacturing taskwith a set of additive manufacturing units 10102.

In embodiments, a scoring system 10634 helps with scoring and ratingvarious entities in the distributed manufacturing network 10130, such asbased on their performance, quality, timeliness, condition, status, orthe like. In embodiments, the scoring system 10634 helps with rating amanufacturing node based on a customer satisfaction score, such as formeeting customer requirements. In embodiments, the scoring system 10634helps with rating an engineer or other worker based on thecondition/performance in completing an additive manufacturing task,including time required, quality of output, energy used, and otherfactors. In embodiments, the scoring system 10634 helps with rating theadditive manufacturing unit 10102 based on the condition or performancein completing an additive manufacturing task, including process metrics,output metrics, product quality measures, economic measures (such asROI, yield, profit and the like), customer satisfaction measures,environmental quality measures, and the like.

In embodiments, an order tracking system 10636 helps with tracking aproduct order through its movement in the distributed manufacturingnetwork 10130 till it is finally delivered to the customer. The ordertracking system 10636 may receive state data from various entities ofthe distributed manufacturing network 10130 on real-time or a nearreal-time basis. For example, a 3D printer may provide updates onproduction stage data or a shipping system may provide updates onproduct location. This information may then be tracked, such as by auser or customer identity, on real time or near real-time basis throughthe order tracking system 10636. A workflow manager 10638 manages thecomplete 3D printing production workflow for the distributedmanufacturing network 10130 including various events, activities andtransactions related to one or more entities of the network 10130.

In embodiments, an alerts and notifications system 10640 providesalerts, notifications or reports about one or more events to a user orcustomer of the network 10130. For example, the alerts and notificationssystem 10640 may receive data related to certain production parametersor errors based on monitoring of the production workflow, based on whichthe alerts and notifications may be generated. Such alerts,notifications, or reports may then be transmitted to a computing device(e.g., a computer, tablet computer, smart phone, telephone, mobilephone, PDA, TV, gaming console and the like) of a user or customer viaemail, text message, instant message, phone call, and/or othercommunication (e.g., using the Internet or other data or messagingnetwork).

In embodiments, the error notifications may provide options for a use ofthe platform 10110 related to continuing or stopping production ormaking adjustments to the design or production settings.

In another example, a user or customer of the distributed manufacturingnetwork may be provided with custom reports including live status andanalytics based on real-time and historical data of the distributedmanufacturing network 10130. In embodiments, the custom report mayinclude data and analytics related to demand, production capacity,material usage, workflow inefficiencies, output type, output parameters,materials used, cost, ROI and the like across one or more manufacturingnodes in the network.

In embodiments, the payment gateway 10642 manages the entire billing,payment and invoicing process for a customer ordering a product usingthe distributed manufacturing network 10130. This may include recordingevents or transactions on an account or ledger, such as a distributedledger, such as a blockchain-based ledger. Payments may be allocatedaccording to a set of rules, such as embodied in a smart contract, suchas to allocate payments across payees; for example, printing from acopyright-protected or other proprietary instruction set may trigger aroyalty payment to the intellectual property owner, manager, or thelike.

It will be apparent that these applications provided by the platform10110 are only presented by way of example and should not be construedas limiting the scope and many other applications may be provided tomanage one or more aspects of the distributed manufacturing network10130.

In embodiments, an authentication application may be provided toauthenticate the identity of users of the platform through one or moreauthentication mechanisms including a simple username/passwordmechanism, biometric mechanism or cryptographic key exchange mechanism.Similarly, an authorization application may define the roles and accessprivileges of users of the platform such that users with different rolesare provided different access privileges. For example, an“administrator” or “host” privilege may allow a user of the platform tomake changes to platform configuration, add and remove programs, accessany files and manage other users on the platform; an “engineer”privilege may allow a user of the platform to operate the platform; anda “service” privilege may allow a user of the platform to access asubset of administrator privileges to perform maintenance and repairactivities.

Some other example applications provided by the platform 10110 forproduction management include part marking, slicing tool selection,alerts and notifications for feedstock supply, printing queuemanagement, printer floor management, job scheduling (including acrossmultiple units), finish work management, packaging management,preparation for logistics, and the like. Some example applicationsprovided by the platform 10110 for production reporting include orderfailure reporting, management information system alerts, remote qualityassurance, certification, indexing and the like. Some exampleapplications provided by the platform 10110 for production analysisinclude order matching, production failure analysis, warranty managementand so on. Some example applications provided by the platform 10110 forvalue chain management include payment processors, digital formatconversion, production restrictions, export restriction filtering, andso on.

In embodiments, the platform 10110 is integrated with one or more thirdparty systems of various types described herein and in the documentsincorporated by reference herein, such as an Enterprise ResourcePlanning (ERP) system 10644, a Manufacturing Execution system (MES)10646, a Product Lifecycle Management (PLM) system 10648, a maintenancemanagement system (MMS) 10650, a Quality Management system (QMS) 10652,a certification system 10654, a compliance system 10656, a Robot/Cobotsystem 10658, an SCCG system 10660 and the like. In embodiments, theplatform is integrated into or a value chain network control towersystem, such as for managing a set of value chain network entities.

In embodiments, an API system facilitates the transfer of data betweenthe platform 10110 and one or more third party systems. The API systemmay consist of a set of APIs for transfer of instruction sets, forpassing alerts, notifications and the like, for transmitting eventstreams (such as workflow-related events), for passing sensor data (suchas process sensing from manufacturing, environmental sensing andothers), for handling user data, for processing payments, forintegrating with smart contracts, blockchains, and other systems, forpassing data with AI systems, for passing data with 3D rendering andother modeling systems, and many others.

In embodiments, the Enterprise resource planning (ERP) system 10644helps streamline and integrate business processes across finance, sales,marketing, service, engineering, product management, accounting,procurement, distribution, resources, project management, riskmanagement and compliance, among other functions, both within amanufacturing node and across multiple manufacturing nodes in thedistributed manufacturing network 10130. ERP System 10644 may tietogether various production and value chain processes in the distributedmanufacturing network 10130 and enable the flow of data between them.

In embodiments, the Manufacturing execution system (MES) 10646 connectsand monitors machines, processes, equipment, tooling and materials tostreamline manufacturing operations both within a manufacturing node andacross multiple manufacturing nodes in the distributed manufacturingnetwork 10130. The MES 10646 may integrate processes spanningproduction, distribution, supply chain, maintenance, quality and laboroperations. Also, the MES 10646 may coordinate with other systems andentities in the distributed manufacturing network 10130 to help withmaking decisions related to advanced planning, production capacityanalysis, inventory turns and lead times.

In embodiments, an additive manufacturing platform, such as thatassociated with a value chain or other network, may be designed,prepared, configured and/or deployed to support the design, development,manufacture and distribution of parts and maintenance materials (e.g.,oil, gas, other chemicals) for vehicles used to distribute products thatmay include trucks, trains, airplanes, boats, drones, etc.; parts andmaintenance materials for machines (e.g., robots) used in packagingproducts; parts and maintenance materials for tools and machines (e.g.,robots) used in moving packaged products from warehouse to vehicles;arts repair on existing parts (and, while in service); missing partsfrom a product that is otherwise ready to go, or some other part orcomponent for the design, development, manufacture and distribution ofparts and maintenance materials.

In embodiments, an additive manufacturing platform, as described herein,may be designed, prepared, configured and/or deployed to support themonitoring of packaging materials (e.g., boxes, crates, wrap material,and the like) and need to generate more “as needed.” The additivemanufacturing platform may address a “recall” situation by adding orrevising a product in-warehouse, and may monitor for problems withvehicles, machines, tools, and other equipment being used and thenreplacing needed parts or materials “as needed,” creating toolson-demand as needed by workers or robots in warehouse/distributionnetwork and the like.

In embodiments, an additive manufacturing platform, as described herein,may be designed, prepared, configured and/or deployed to supportprocessing manufacturing inputs, such as using an artificialintelligence system (e.g., a robotic process automation system trainedon a training set of expert service visit data), to determine arecommended action, which in embodiments may involve replacement of apart and/or repair of a part, or some other activity. In embodiments,the additive manufacturing platform may automatically determine that anelement should be additively manufactured to facilitate repair, such aswhere a complementary component may be generated to replace a worn orabsent element. In example embodiments, some techniques and/ortechnologies that may be utilized with the warehouse/distribution centermay include, but are not limited to: providing and/or including multiplesource materials to generate in real time (i.e., on the fly) differenttools, parts, and/or packaging; using AI to optimize product design,manufacturing process configuration (including packaging materialgeneration process), job scheduling, prioritization and/or logistics(efficiency of warehouse processes for replacing parts, materialswithout disrupting other general processes involved inwarehouse/distribution center); enriching AI with input/source/trainingset data relevant to design factors, economic factors, quality factors,etc. involved in particular example embodiments (e.g., using sensors andmonitoring of data to adjust manufacturing processes of parts materialsneeded for machines and/or packaging materials); coupling inputs,process data and outputs with digital twins for running simulations ofindividual processes or a combination of processes to anticipatematerial needs for being able to produce or manufacture tools, parts,packaging, and/or fix machines with materials in real time (as needed);networking additive manufacturing nodes in meshes and/or fleets forcoordinated operation within a warehouse/distribution network in anefficiency manner with respect to producing tools, parts, packaging,and/or other materials used to fix machines in real time; using robotsthat are able to attach to machines and then print directly onto aproduct, print tool, print parts for machines used inwarehouse/distribution network, print packaging, and/or print materialsused to fix machines in real time; using hybrids/pairs of differenttypes of 3D print additive manufacturing including any and all of theitems listed within warehouse/distribution center network processes forfixing products, producing tools, producing parts, producing packaging,and/or producing other materials to fix machines in real time (asneeded).

In embodiments, the Product Lifecycle management (PLM) system 10648helps manage the part or product across the entire lifecycle, fromconception and design through manufacturing and distribution to customeruse and service. The PLM system 10648 may contain accurate, real-timeproduct information across the lifecycle and value chain. This helpswith developing and managing the product in a manner that is responsiveto feedback from one or more distributed manufacturing network entities,such as customers using the product, distributors, logistics providers,regulators, safety professionals, service professionals, salespeople,product managers, designers, resellers, and many others. This may alsoenable an accelerated proof of concept and rapid customization of theproduct in the product development stage. Also, this may help withpredicting product demand and prices, improving customer engagement,performing product testing while in customer use and providingpre-emptive warranty management.

In embodiments, the maintenance management system (MMS) 10650 monitors aset of 3D printers, cutting tools, filters, machine lasers and othermachines, manages spare parts, maintains records and uses artificialintelligence and machine learning models to efficiently self-diagnosemaintenance requirements and generate work orders. In embodiments, theMMS 10650 monitors a set of other machines, equipment, products,fixtures, or other assets, maintains records, and manages maintenanceoperations for that set of items, including coordinating additivemanufacturing workflows (such as to produce spare parts, tools,workpieces, accessories, replacement elements, and the like) with othermaintenance workflows. In embodiments, this occurs with automation, suchas robotic process automation, such as where an RPA agent is trainedupon a set of expert interactions to undertake, or to support,operations performed by maintenance workers.

In embodiments, the Quality Management system (QMS) 10652 determineswhether a printed part has been produced correctly by comparing realtime sensor data with expected feedback data wherein the expectedfeedback data is generated from at least one of historical data, testdata, and machine learning. In embodiments, the QMS 10652 also generateswarranty certification including the duration of part warranty and scopeof coverage upon determining completion of testing and qualityassurance.

In embodiments, the QMS 10652 includes automated part metrology andutilizes a vision system with variable focus optical system andartificial intelligence-based pattern recognition for automated partmetrology. In embodiments, the vision system may include a conformablevariable focus liquid lens assembly and a processing system thatdynamically learns on a training data of outcomes, parameters and datacollected from the conformable variable focus liquid lens assembly totrain an artificial intelligence system to recognize an object. Theconformable variable focus liquid lens assembly may constantly adjustbased on environment factors and on feedback from the processing systemto generate training data that is deeper in context and that correspondsto the physical light that the image represents. By training the visionsystem to recognize objects using variable optical parameters throughthe liquid lens assembly, the processing system may learn about the mostoptimum optical setting to detect an object. The vastly more dynamicinput to the vision system may result in creating a richer context andproviding superior object recognition.

In embodiments, the certification system 10654 is configured to generateworkflow and process control documentation to obtain certificates ofconformance from one or more Manufacturing Certification Authorities orStandards Authorities. In embodiments, the one or more ManufacturingCertification Authorities or Standards Authorities include InternationalOrganization for Standardization (ISO), European Certification (CEmarking) bodies, Underwriters Laboratories (UL), Society of AutomotiveEngineers (SAE), Federal Aviation Administration (FAA), TUV SUD, DNV GL,AS9100, IAQG 9100, American Society of Testing and Materials (ASTM),NIST (research, measurement science and standards), Fraunhofer Institute(research) and Sandia National Labs (research).

In embodiments, the compliance system 10656 configured to performcompliance checks on 3D printed parts. In embodiments, compliancechecking occurs by or with support from robotic process automation, suchas where a compliance model or algorithm is trained by qualified expertsin certification/compliance with a specific requirement on a trainingset of compliance review data or the like. In embodiments, a set ofdomain-specific or topic-specific models may be trained, such as one foreach compliance domain or topic, such as for compliance withenvironmental standards, material standards, structural standards,chemical standards, safety standards, electrical standards, fire-relatedstandards, and many others.

In embodiments, robot/cobot system 10658 may include an autonomousrobotic system or arm unit integrated with a set of additivemanufacturing units 10102. For example, the additive manufacturing unit10102 may be contained within the housing or body of a robotic system,such as a multi-purpose/general purpose robotic system, such as one thatsimulates human or other animal species capabilities. Alternatively, oradditionally, the additive manufacturing unit 10102 may be configured todeliver additive layering from a nozzle that is disposed on an operatingend of a robotic arm or other assembly.

In embodiments, the autonomous additive manufacturing platform 10110 maycreate and manage profiles of different distributed manufacturingnetwork entities. For example, profiles may include, without limitation:a part or component profile with accompanying part data structures maystore part-related information and component-related information,including name, number, class, type, material(s), size, shape, function,performance specifications and the like; a batch profile withaccompanying batch data structures for storing batch-related informationincluding batch number; batch date, bin number, batch type, locationinformation (such as origin), batch inspection data, and the like; amachine profile with accompanying machine data structures for storingmachine related information including identifier, name, class, functionetc.; a manufacturing node profile with accompanying manufacturing nodedata structure for storing information related to manufacturing nodeincluding identifier, location, order history, production capacity, andprevious product designs; a packager profile with accompanying datastructures for storing packaging related information; a user profilewith accompanying user data structures for storing user relatedinformation; and a behavioral profile with accompanying data structuresfor storing behavioral information, among many others. Some examples ofusers of the platform 10110 may include a designer looking to generate adesign for fabrication; an engineer looking to print and manufacture apart; a CFO looking to optimize price for production; or a customerlooking to get a product printed. Users may include role-based users,such as described in connection with other use cases referenced hereinand in the documents incorporated herein by reference, such as varioususers described in connection with digital twins, such as executive andother role-based digital twins, consumers of automatically generateddata stories, and many others.

The metal additive manufacturing platform 10110 described herein mayhelp in automating and optimizing a very wide range of manufacturing andvalue chain functions. Some examples of such functions include processand material selection, feedback formulation, design optimization, riskprediction and management, sales and marketing, coordination with supplychain and logistics workflows (including reverse logistics and returns)for manufactured products and/or related items or services (such asparts, accessories or the like, among others), maintenance workflows,recycling workflows and customer service. FIG. 119 is a schematicillustrating an example implementation of the platform 10110 forautomating and managing manufacturing functions and sub-processesincluding process and material selection, hybrid part workflow,feedstock formulation, part design optimization, risk prediction andmanagement, marketing and customer service according to some embodimentsof the present disclosure.

Process and Material Selection

The selection and use of one or more processes or materials for additivemanufacturing may be automated and optimized. The platform 10110 maytake as input the product requirements in terms of part properties,price, performance characteristics etc. and automatically determine theprocesses or material for building the part. The artificial intelligencesystem 10212 may consume model information comprising physical, chemicaland/or biological model of material behavior, including structural,stress, strain, wear, load bearing, response to contamination, chemicalinteraction with other materials, interaction with biological elements(antibacterial, antiviral, toxicity), etc. The artificial intelligencesystem 10212 may then automate and optimize process and materialselection, including based on expert feedback and/or feedback fromtrials/outcomes.

Referring now to FIGS. 115, 116, and 119 an example embodiment forautomating process and material selection is described.

A part design comprising model information and product requirements ispresented to the design and simulation 10116 where it is evaluated formanufacturing compatibility with at least one type of the additivemanufacturing unit 10102 in the manufacturing node 10100. The design andsimulation 10116 may be assisted by the artificial intelligence 10212,the simulation management 10514, the printer twin 10506 (which inembodiments may be a twin of any type of additive manufacturing unit)and the process and material selection twin 10702 for performing theoptimization. An example analysis includes the use of the printer twin10506 in the digital twin system 10214 to simulate and compare partdesign dimensions and accuracy with available 3D printer workingenvelopes and specifications.

After a part design is validated to be compatible with one or more ofthe additive manufacturing units 10102 in the manufacturing node 10100,part data for manufacturing may be optimized for export at the designand simulation 10116. For example, an optimized STL file may be producedfrom a finely meshed 3D CAD surface model to meet part accuracyrequirements, and then exported to the autonomous additive manufacturingplatform 10110.

The autonomous additive manufacturing platform 10110 may include aprocess and material selection system 10704. Using optimized part datafrom the design and simulation 10116, external information includingpricing and market related information from sources such as the valuechain entities 10126, and help from the artificial intelligence system10212, the process and material selection system 10702 performs analysisto select one or more of the additive manufacturing units 10102 for partmanufacturing. In one example, the process and material selection system10702 may analyze availability and cost of printer feedstock materialsto select the additive manufacturing unit 10102 that manufactures thepart according to specifications while optimizing for the lowest cost ofmanufacture.

Referring to FIGS. 116, 118 and 120 , when manufacturing is complete,part and process data related to the outcome of the 3D printing processis collected by the data collection and management system 10202. Outcomedata is provided to the machine learning system 10210 along withsimulation, external, and training data to train or improve the initialmachine learning model 10213.

The following is an example of autonomous design validation andselection of a 3D printing process and material. Referencing FIGS. 114and 115 , part design data is entered at user the interface 10112 and isthen provided as input to the design and simulation 10116 for partvalidation. The part design data provided at the user interface 10112may include the following part specifications and order requirements: Aform or shape described by a 3D CAD solid model; Use-case loading asapplied to the provided 3D CAD model; Part design stress factor ofsafety: >2; Maximum part weight; Corrosion requirement: Compatibilitywith seawater and salt spray; Order part quantity 10; and Delivery time.

With help from the artificial intelligence system 10212, the design andsimulation 10116 performs multiple screening analyses as follows: amaterial analysis that identifies titanium, Inconel, and 316 stainlesssteel as materials that meet corrosion requirements; a materialanalysis, assisted by simulations from the printer twin 10506 and theprocess and material selection twin 10702, that identifies powder bedfusion or metal material extrusion as 3D printing processes that matchavailability of the additive manufacturing units 10102; a stress andweight matrix analysis calculated for part geometry and loading thateliminates Inconel and 316 stainless steel due to weight considerations,but qualifies titanium for both weight and maximum stress. Followingcompletion of the screening analysis, process and selection system 10704is used to complete final additive manufacturing unit 10102 selectionfrom the subset of additive manufacturing units 10102 available formanufacturing.

Hybrid Part Workflows

The selection and use of one or more hybrid manufacturing workflowsoptimized for applying additive material on existing parts may beautomated to produce a modified part assembly. Hybrid part workflows canbe used to develop new manufacturing processes, repair existing parts,and modify existing parts to improve value chain outcomes.

The autonomous additive manufacturing platform 10110 may take as inputexisting and OEM part information comprising physical, chemical,manufacturers specifications, etc., including information based onexpert feedback and/or feedback from trials/outcomes. The AI system10212 uses input data to help with automatic validation of a part forone or more hybrid workflows in the workflow management applications10208.

In a part repair example, data from the user interface 10112 and thedata sources 10114 are provided to the design and simulation 10116.Example data includes a combination of measurements and expertobservations and/or OEM part information such as specifications and CADmodels. The design and simulation system 10116 analyzes part dimensionaland material repair requirements with reference to their compatibilitywith at least one type of additive manufacturing unit 10102 in themanufacturing node 10100. The design and simulation 10116 may beassisted by the artificial intelligence 10212, the simulation management10514, and the digital twin systems 10214, for example, analyses mayinclude the use of the printer twin 10506 and the part twin 10504 in thedigital twin system 10214 to simulate modified part manufacturingoutcomes using available 3D printer capabilities or determinecompatibility of OEM part material with available 3D printer materials.

After a modified part is validated by the design and simulation 10116 tobe compatible with one or more of the additive manufacturing units 10102in the manufacturing node 10100, modified part data is exported to theautonomous additive manufacturing platform 10110 where the process andmaterial selection system 10704 selects one or more of the additivemanufacturing units 10102 for manufacturing using one or more hybridworkflows. Example hybrid workflows include the build-up of worn partareas or replacement of chipped or cracked areas of parts.

Referring to FIGS. 118 and 119 , when modified part manufacturing iscomplete, part and process data related to the outcome of the 3Dprinting process is collected by the data collection and the managementsystem 10202, where data comprising modified part parameters,measurements, and so on can be exported to systems responsible formanaging warranty, safety, and related compliance, for example the ERPsystem 10644, the certification system 10654, the compliance system10656, etc. In embodiments, data may be used to set parameters for asmart contract, such as populating warranty-related, safety-related,liability-related, or other terms of a smart contract. The platformand/or smart contract may store the data in a blockchain.

In embodiments, hybrid manufacturing workflows may be used to modify anexisting part design to produce a new design, for example whenincorporating new functional or safety features that improve partperformance.

In embodiments, hybrid manufacturing workflows may be used to producenew parts comprising multiple materials that may require more than one3D printer or 3D printing process to produce targeted part or productcharacteristics.

Referring to FIGS. 114 and 115 , in embodiments, hybrid manufacturingworkflows may specify and manage specialized pre-processing 10104 andpost-processing 10106, for the additive manufacturing unit 10102manufacturing. Examples include part cleaning, machining, grinding,surface finishing, etc. to enable 3D printing, or to produce modifiedparts that meet original equipment part specifications.

Feedstock Formulation

The selection, purchase, and management of 3D printer feedstock may beautomated and optimized to improve manufacturing efficiency, controlsupply chain logistics and cost, and to provide new part productioncapabilities.

Referring now to FIG. 119 , a feedstock formulation system 10706, helpedby the artificial intelligence 10212 and a feedback formulation twin10708, automatically formulates and adjusts 3D printer feedstockaccording to production requirements, supply chain conditions, pricingand availability information or other data. For example, the feedstockformulation system 10706 may select commercially available feedstocksuch as Ni Alloy 718 from GE Additive, or suggest local manufacture ofan equivalent material at lower cost from commercially availableconstituent materials. In embodiments, pricing and availabilityinformation may be managed by processing, such as by an API of theplatform and/or the feedstock formulation system, a set of the terms andconditions of a set of smart contracts, such as smart contracts thatprovide current and/or future (e.g., in a spot market at designed timesin the future) pricing information, availability information (includingby volume, by time and by delivery location) for various classes offeedstock materials, including by material type, material quality (e.g.,where there are varying grades of the material that can be purchased asfeedstock), or other properties (such as material origin (e.g.,reclaimed from recycling or other sustainable sources, mined withsustainable practices, purchased from ethical sources, and the like)).In embodiments, the platform may aggregate availability information,pricing and the like across multiple smart contracts or a blend of smartcontracts and other sources (e.g., offers that are placed in theplatform by data entry and/or API) to provide an aggregated feedstockavailability data structure upon which the system may operate, such aswhere feedstock may come in lots or batches from different suppliers,places of origin and the like. The platform may automatically generate afeedstock purchasing plan, which may include a set of current purchases,purchases of options or futures, and a plan for future purchases. Inembodiments, the platform may automatically modify the feedstockpurchasing plan based on changes in conditions, such as needs (e.g.,where production varies relative to plan and/or demand varies relativeto plan), pricing (of end products and/or materials), availability, andthe like. This may occur using artificial intelligence, such as byrobotic process automation trained on a training set of feedstockpurchasing management data, which may use any of the machine learning orother artificial intelligence techniques described herein, includingsupervised, semi-supervised and/or deep learning. The artificialintelligence system may further adjust a set of contract terms andconditions for feedstock purchasing according to the modified plan, suchas by operating on a set of smart contracts via their APIs or otherinterfaces and/or by providing a set of recommendations for execution bya user or a hybrid of a user and an intelligent agent or otherartificial intelligence system.

In embodiments, the feedstock formulation system 10706 may formulate oneor more custom feedstocks with help from the machine learning system10210, the artificial intelligence system 10212, the machine learningmodel 10213 for feedback formulation, the simulation management system10514, and the feedstock formulation twin 10708. The machine learningsystem 10210 may train a model using feedstock data that may be storedin a feedstock datastore, such as a graph DB that organizes differentfeedstocks according to performance properties. The simulationmanagement system 10514 may run simulations using the feedstockformulation twin 10708 to vary feedstock properties and to record theoutcome of each simulation. In embodiments, printer twin 10506 may alsobe used to simulate and compare future manufacturing outcomes whenvarying feedstock formulation.

Referring to FIGS. 116 and 119 , the feedstock formulation system 10706works with the artificial intelligence system 10212, and the machinelearning system 10210. A combination of training, manufacturing outcome,and external data such as pricing and availability information andexpert and customer feedback is collected at the data collection andmanagement system 10202, where it is used to train or improve theinitial machine learning model 10213 for feedback formulation.

Referring now to FIGS. 114, 115 and 119 , in embodiments, the feedstockformulation system 10706. may include a physical subsystem that isintegrated with the manufacturing node 10100 and one or more of theadditive manufacturing units 10102. This physical subsystem of thefeedback formulation system 10706 may be managed by the autonomousadditive manufacturing platform 10110. The manufacturing workflowmanagement applications 10208 may include an application that routesfeedstock material as necessary, and the data collection and managementsystem 10202 may provide feedstock inventory levels. The feedstockformulation system 10706 may include one or more automated productionand transport systems that deliver feedstock material and performfeedstock material changes for the additive manufacturing unit 10102.

Design Optimization

Optimizing part design for use with additive manufacturing processestypically requires special software, equipment, training, technicalknowledge, and the ability to provide and interpret process data andmanufacturing outcomes. Autonomous or guided product design can be usedto improve value chain outcomes by using pre-engineered part librariesor expert systems to provide either autonomous part design, orexpert-assisted designs that are optimized for metal additivemanufacturing processes. Resulting workflow and process functionalitymay be further optimized by incorporating limitations or recommendationsbased on real-time analysis of value chain entities that provide data onthe availability of a selected material or 3D printer, part cost anddelivery time, and so on.

Referring to FIG. 118 , part design optimization for 3D printingprocesses may be automated using the design and simulation 10116, wherepart function and/or class criteria are organized in a design library10618 and used to guide or fully automate part design for manufacturing.Part functions and classes have inherent minimum design criteria imposedby standards, best practices, engineering experts, and so on. Partfunction examples include a self-lubricating bearing made from sinteredmetal that must meet chemical, mechanical, and other properties found inthe ISO 5755 standard, or an electrical hand tool where materials mustmeet 1000V electrical insulation standards found in the IEC 60900standard. Part classification examples include parts for use inexplosive atmospheres, where materials of construction must benon-sparking, or parts for medical tools used in surgery, wherecorrosion characteristics must comply with the ASTM F1089 standard.

Referring to FIGS. 115, 116, 118, and 119 , in one example embodiment, anew part request that has a specific function is received by the userinterface 10112 and communicated to the design and simulation 10116,where the design libraries 10618 are searched for tested and viable 3Dprinted part models that match part function. In embodiments, one ormore parts from the design library 10618 are recommended to the user,such as via the interface 10112, as a design recommendation or guidance.In embodiments, design libraries may also include product assemblies,wherein completed assemblies and all parts in the assembly meetfunctional or class criteria.

In embodiments, one or more candidate parts are automatically selectedby a design optimization system 10710. With help from the machinelearning system 10210 and the artificial intelligence system 10212, thedesign optimization system 10710 optimizes the part design and submitsthe same to the autonomous additive manufacturing platform 10110 formanufacturing.

In embodiments, the design optimization system 10710 may use machinelearning models trained by product design experts. In embodiments, thedesign optimization system 10710 may use machine learning models trainedusing data of prior designs and their outcomes.

In embodiments, the design optimization system 10710 may use agenerative or evolutionary approach to design. The system may start withdesign goals and then explore innumerable variations by addingconstraints before selecting a final design based on evolutionarymodels. The evolutionary models are based on the principle of naturalselection, such as where the most optimal designs are selected fromamong an initial population of potential designs through a series ofevolutionary stages. Generative models may include models like DALL-E™that mix visual and text-based artificial intelligence systems, as wellas further hybrids for generating visual, 3D, text, color, texture,strength, flexibility, and many other properties, including usingspecialized artificial intelligence systems for generating variations ofeach of a large set of properties and generating combinations, such aspairs, triplets, and higher-order n-tuples of properties. Inembodiments, generative models may generate and/or select designinstance that represent combinations of properties that are shared amongsemantically distinct objects or topics, such as a cat and basket inorder to produce and/or select a set of designs that embody the sharedset of properties.

In embodiments, evolutionary models may be based on genetic algorithms(GA), evolution strategy (ES) algorithms, evolutionary programming (EP),genetic programming (GP), and other suitable evolutionary algorithms. Inembodiments, the evolutionary models may use various feedback andfiltering functions, such as ones based on semantic properties, onesbased on design constraints (such as acceptable color palette forbrand), ones based on physical or functional requirements, ones createdby consumer engagement (such as surveys, engagement tracking and/or A/Btesting), ones based on outcomes (such as sales, profits, or others),ones based on cost (of materials, manufacturing, logistics, or others),ones based on safety or liability, ones based on regulatory requirementsor certification, and many others. In embodiments, feedback to designevolution is taken from a set of smart contracts, such as a set of smartcontracts that offer various design variations for purchase,reservation, or the like. For example, a design may be evolved based onfavorable smart contract engagement, such as where a particular designis reserved via the set of smart contracts at a profitable price and infavorable volumes.

In embodiments, an evolutionary design system coupled to a set ofadditive manufacturing units 10102 continuously offers a set of productsvia smart reservation contracts by which users may reserve units formanufacturing according to the offered designs, such that the capacityof the additive manufacturing system is continuously engaged in evolvingthe designs to provide the most favorable outcomes in the smartcontracts (based on measures of profitability, for example) and sellingthe products to the users who reserved them via the smart contracts.Smart contract parameters, including prices, terms of delivery and thelike, may be automatically adjusted, such as to account for time tomanufacture, logistics factors, and the like. The system may beconfigured to integrate with an e-commerce system, such as to offerproducts on a marketplace, an auction site, a mobile application, or thelike, as well as with other environments where purchasing is enabled,such as on-site systems (kiosks), in-game transaction environments,AR/VR environments, smart displays, and many others.

Referring to FIG. 116 and FIG. 119 , when manufacturing is complete,part and process data related to the outcome of the 3D printing processis collected by the data collection and management system 10202. Outcomedata is provided to the machine learning system 10210, as feedback alongwith simulation, external, and training data to train or improve thelearning model 10213.

Risk Prediction and Management

Referring now to FIG. 119 , a risk prediction and management system10712 interfaces with, links to, or integrates the artificialintelligence system 10212. In example embodiments, the risk predictionand management system 10712 may be configured to predict and manage riskor liability with respect to manufacturing, delivery, utilization and/ordisposal of a part, product or other item by the distributedmanufacturing network 10130, among other risks or liabilities.

In embodiments, the machine-learning system 10210 trains one or more ofthe models 10213 that are utilized by the artificial intelligence system10212 to make classifications, predictions, and/or other decisionsrelating to risk management, including for parts and productsmanufactured by the distributed manufacturing network 10130 and for thesystems, workflows, and other activities in which they are involved.

In example embodiments, the model 10213 may be trained to predict riskof part failure by detecting the condition of a part. The machinelearning system 10210 may train the model using part data and one ormore outcomes associated with the part condition, such as on a trainingset of data on outcomes of similar parts, similar materials, and thelike, including historical data on wear-and-tear during usage,historical data on material deterioration under various ambient orenvironmental conditions, data on defects or faults discovered duringinspection or reported by customers or others, and other data sources.Part data may include any of the attributes or parameters notedthroughout this disclosure and the documents incorporated by referenceherein, such as part material, part properties, manufacturing date,material supplier, part specifications and the like. In this example,outcomes used to train the machine learning system 10210 to predictrisk, failure of liability may include projected outcomes from models,such as scientific models of various types described throughout thisdisclosure and the documents incorporated by reference herein (e.g.,physics, chemistry, biology, materials science, and others), economicmodels, and many others, which in embodiments may be embedded into adigital twin system, such as to model whether a part twin 10504, producttwin, or other twin is in a favorable operating condition during orafter simulation of a set of events, a passage of time, or the like. Inthis example, one or more properties of the part twin 10504 are variedfor different simulations and the outcomes of each simulation may berecorded. Other examples of training risk prediction and managementmodels may include the model 10213 that is trained to optimize productsafety, a model that is trained to identify parts with a high likelihoodof failure, and the like.

In example embodiments, the model 10213 may be trained to predict riskof non-delivery of a product to a customer, such as due to supply chainand other disruptions, such as ones caused by various external eventslike equipment failures, strikes and other labor disruptions, bordercontrol activities (such as customs inspections, travel bans andothers), limits on shipping, traffic congestion, power outages, stormsand other natural disasters, catastrophes, economic disruptions (such aslarge changes in tariffs), regulatory changes (such as bans on import orexport or changes in where products may be legally sold or used),pandemics, political unrest and the like. In this example, a model maybe trained to predict supply chain disruption by discovering,extracting, transforming, normalizing, processing, and/or analyzing datafrom one or more external sources like social media feeds, weatherpatterns, news feeds, websites (e.g., websites providing contentrelevant to the above, marketplace web sites, research websites, andothers), crowdsourcing systems (which may include posing queries orprojects to crowds in order to solicit input on specific factors, suchas economic factors, behavioral factors, trends and the like),algorithms (such as ones trained to provide specific predictions ofevents), and many others. The artificial intelligence system 10212 maythen predict and assess the impact of the predicted disruption to decideif a supply chain redesign may be required to minimize the disruption.Impact assessment and/or prediction may use a set of economic, financialor operating models, among many others, such as to assess primary,secondary, and other effects on an overall workflow or system. Forexample, assessment or prediction may include the impact of the absenceof a component on the ability to deliver a system on time; the impact ofdiminished or late supply on sales (e.g., missing a seasonal windowhaving major impact on product demand for some products, like Halloweencostumes or beach chairs); the impact of diminished or late supply onpricing (such as where anticipated shortages may dictate a need for aprice increase and/or purchasing limits to balance supply and demand andavoid shortages or outages or products); the impact on contractliability (such as liability for failure to deliver, including theobligation to pay for the cost of the buyer to cover in the marketplaceby buying substitute items); the impact on brand or reputation; and manyothers.

In embodiments, the artificial intelligence system 10212 may leverage anenvironment twin 10714, the manufacturing node twin 10510 and/or othertwins to run a set of simulations to assess the impact of the disruptionon one or more manufacturing nodes. The risk prediction and managementsystem 10712 may then initiate a supply chain redesign or productresupply event to minimize the impact of the disruption. Furthermore,the outcomes of such an event (e.g., improved lead time) may be reportedto the machine learning system 10210 to reinforce the model used to makethe decisions.

Marketing and Customer Service

Referring now to FIG. 119 , a marketing and customer service system10716 interfaces with, links to, or integrates the artificialintelligence system 10212. In example embodiments, the marketing andcustomer service system 10716 may be configured to provide personalizedsales, marketing, advertising, promotion and/or customer service withrespect to a product or other item provided by the distributedmanufacturing network 10130.

In embodiments, the machine-learning system 10210 trains one or more ofthe models 10213 that are utilized by the artificial intelligence system10212 to make classifications, predictions, and/or other decisionsrelating to sales, marketing, advertising, promotion and/or customerservice for products manufactured by the distributed manufacturingnetwork 10130.

In example embodiments, the model 10213 may be trained to predictbehavior and purchase patterns of one or more customers to providepersonalized sales, marketing, advertising, promotion and/or customerservice. In embodiments, the machine learning system 10210 may train themodel using customer data and one or more outcomes associated withcustomer response to a personalized campaign, such as using various datasources that provide insight into consumer sentiment, behavior, or thelike, including search engines, news sites, websites, behavioralanalytic systems and algorithms, consumer sentiment measures,microeconomic measures, macroeconomic measures, and many others. A modelmay be seeded with various economic, behavioral, and other models,including demographic, psychological, economic, game theoretic,cognitive, and other models. Customer data may include any of the typesdescribed throughout this disclosure and the documents incorporated byreference herein, such as identity data, transactional and payment data,location data, demographic data, psychographic data, location data,wealth data, income data, sentiment data, affinity data, loyalty programdata, clickstream data (including interactions with social media,applications, websites, mobile devices, AR/VR systems, video games,entertainment content and other digital content), point-of-sale data,in-store behavioral data (such as path tracing data within stores, dwelltimes associated with particular types of products, and the like), brandloyalty data, shopping data, search engine data (such as search topicsinvolving shopping), social media footprint, purchase history, loyaltyprogram data and many others. The customer twin 10718 may capture a setof customer responses to a marketing or advertising campaign or one ormore product recommendations, offers, advertisements or othercommunications by tracking outcomes like customer attention or actions(including mouse movements, mouse clicks, cursor movements, navigationactions, menu selections, and many others) measured through a softwareinteraction observation system, or purchase of a product by a customer.In this example, one or more parameters of the marketing or advertisingcampaign may be varied for different simulations of a customer twin andthe outcomes of each simulation may be recorded.

In embodiments, the marketing and customer service system 10716 mayinterface with the artificial intelligence system 10212 to providepersonalized sales, marketing, advertising, promotions and/or customerservice, including providing personalized marketing and advertisingcampaigns and providing product recommendations. In embodiments, theartificial intelligence system 10212 may utilize one or more of themachine-learned models 10213 to determine a product recommendation. Inembodiments, the simulations run by the customer twin 10718 may be usedto train the product recommendation machine-learning models. In each ofthese examples, a campaign communication, recommendation, or the likemay involve a product or other item that can be manufactured by theadditive manufacturing unit 10102 with a set of attributes that aretailored to the customer and that can be delivered to a designated siteof the customer within a designated time frame at a proposed price.Customization of the offer/recommendation may include providing a designof a product or part to include attributes favored by the customer,including functional attributes, preferred materials (such as to matchmaterials of products already owned by the customer), preferred colors,preferred shapes, and many others. In embodiments, customization mayreference an understanding of products already owned by the customer,such as based on purchase history information, such as where arecommended product can be configured to work as part of a family ofproducts, such as by recommending a product that has compatible color,shape, size, material type, connectivity (e.g., to work as part of aconnected set of products), communication protocol, logo, or the like.

In embodiments, the additive manufacturing platform 10110, such as thatassociated with a value chain network may be prepared, configured and/ordeployed to support printing of personalized entertainment props,backdrops and other items at theme parks, cruise ships, theater and filmproductions and/or other entertainment venues. For example, inconnection with a cruise ship, the additive manufacturing unit 10102 maybe designated to support the printing of cabins, themed rooms orfurniture to fit based on a given theme. The customers may provide theirpreferences in terms of room layout and design, furniture andaccessories, which can be dynamically printed. Similarly, for themeparks the additive manufacturing unit 10102 may be designated to supportthe printing of rockwork, rides and other attractions and for theaterand film productions, movie props, costumes, sets, artifacts and otheraccessories may be custom printed.

In embodiments, the platform may take inputs from or related to theentertainment venue owner, such as inputs indicating the item beingprinted (e.g., technical specifications, CAD designs, or the like);inputs indicating requirements (such as a need to improve an existingroller coaster attraction with custom rockwork, a need to build adinosaur replica, or the like); and inputs captured by cameras,microphones, data collectors, sensors, and other information sourcesassociated with the entertainment venue.

In embodiments, that recommend or configure instructions for additivemanufacturing, the platform 10110 may discover available materialsincluding fabrics, metals plastics etc., configure instructions, andinitiate additive manufacturing, and provide updates to the owner of theentertainment venue, such as updates as to when an element will be readyto use. The platform 10110 may, in some such embodiments, automaticallydetermine, such as by using the artificial intelligence system 10212,trained on an expert data set, and the like, whether a suitable item isreadily available and/or whether use of an additive manufacturing systemto produce the item(s) can reduce delay, to save costs, or the like.

In embodiments, the platform 10110, such as through a trained AI agent,may automatically configure and schedule a set of jobs across a set ofadditive manufacturing units 10102 with awareness of the status of otherrelevant entities involved in other workflows, such as what other workis being done (e.g., to allow for appropriate sequencing of additivemanufacturing outputs that align with overall workflows), the priorityof the printing job (e.g., whether it relates to film scene being shot),the cost of downtime, or other factors. In embodiments, optimization ofworkflows across a set of additive manufacturing entities may occur byhaving the artificial intelligence system 10212 undertake a set ofsimulations, such as simulations involving alternative schedulingsequences, design configurations, alternative output types, and thelike. In embodiments, simulations may include sequences involvingadditive manufacturing and other manufacturing entities (such assubtractive manufacturing entities that cut, dye, or the like and/orfinishing entities that sew, configure, add customer initials, or thelike), including handoffs between sets of different manufacturing entitytypes, such as where handoffs are handled by robotic handling systems.In embodiments, a set of digital twins may represent attributes andcapabilities of the various manufacturing systems, various handlingsystems (robotic systems, arms, conveyors, and the like, as well ashuman workforce) and/or the surrounding environment.

It will be apparent that the above decisions related to predictions,optimizations using the artificial intelligence system 10212 of platform10110 are only presented by way of example and should not be construedas limiting. There may be many other use cases including decisionsrelated to prediction and optimization of pricing by a CFO twin 10720;decisions related to new product launch by a CEO twin based onbehavioral patterns and market trends; and the like.

In embodiments, the autonomous additive manufacturing platform 10110enables the distributed manufacturing network 10130 by managing theproduction workflows within and across one or more manufacturing nodes,thereby facilitating collaboration across the manufacturing nodesthrough the sharing of resources, capabilities and intelligence. Inembodiments, the manufacturing nodes may collaborate for forecasting andprediction of material supply and product demand. In embodiments, themanufacturing nodes may collaborate for design and product development.In embodiments, the manufacturing nodes may collaborate formanufacturing and assembling one or more parts of a product. Inembodiments, the manufacturing nodes may collaborate for distributionand delivery of manufactured products.

The distributed manufacturing network 10130 may thus provide“manufacturing as a service” by leveraging unutilized capacity of one ormore 3D printers by exposing the capacity to one or more users/designersseeking to fabricate 3D printed parts.

In embodiments, a method for facilitating the manufacture and deliveryof a 3D printed product to a customer using one or more manufacturingnodes of the distributed manufacturing network 10130 includes receivingone or more product requirements from the customer; determining one ormore manufacturing nodes, processes and materials based on the productrequirements; generating a quote including pricing and deliverytimelines; and upon acceptance of the quote by the customer,manufacturing and delivering the 3D printed product to the customer.

In embodiments, the product requirements may be a 3D printinginstruction set including a file (e.g., a CAD file and/or an STL file)and any accompanying instructions for printing the product defined inthe file.

In embodiments, the distributed manufacturing network may be implementedthrough a distributed ledger system integrated with the digital threadfor storing a set of entities, activities and transactions related tothe distributed manufacturing network.

In embodiments, a smart contract system may communicate with thedistributed ledger system and may be configured to implement and managea smart contract via the distributed ledger. The smart contract may bestored in the distributed ledger and may include a triggering event. Thesmart contract may be configured to perform a smart contract action inresponse to an occurrence of the triggering event. The distributedmanufacturing network may be configured to receive from a user aninstance of the 3D printing instruction set. The 3D printing instructionset may be tokenized such that the instance of the 3D printinginstruction set can be manipulated as a token on the distributed ledger.The tokenized 3D printing instruction set may be stored via thedistributed ledger. Commitments of various parties (distributedmanufacturing network entities) to the smart contract may be processed.The use of smart contracts in the distributed manufacturing networkhelps in automating the distributed manufacturing workflow.

In embodiments, the distributed manufacturing network facilitates thecreation of a distributed manufacturing marketplace or exchange forbuying and selling of additive manufacturing parts, products andinstruction sets with the manufacturing nodes constituting the sellersand customers constituting the buyers.

In embodiments, the distributed manufacturing network facilitates thecreation of a data marketplace for selling of operational additivemanufacturing data by manufacturing nodes to data aggregators. Inembodiments, the data marketplace is built on a distributed ledger andmanufacturing nodes are compensated using digital token via smartcontracts. In embodiments, the data is anonymized to hide the identityof manufacturing nodes that own the data.

FIG. 120 is a diagrammatic view of a distributed manufacturing networkenabled by an autonomous additive manufacturing platform and built on adistributed ledger system according to some embodiments of the presentdisclosure.

The distributed manufacturing network 10130 is implemented with adistributed ledger system where the distributed ledger may bedistributed at least in part over nodes of the distributed manufacturingnetwork 10139 and may include blocks linked via cryptography. Thedistributed ledger system stores data related to a set of entities,activities and transactions in the distributed manufacturing network10130.

The different manufacturing nodes 10100, manufacturing node 10128,manufacturing node 10800 and manufacturing node 10802 each represent anode in the distributed manufacturing network 10130. Also, the differentsystems within a manufacturing node including the additive manufacturingunit 10102, the pre-processing system 10104, the post-processing system10106, the material handling system 10108, the autonomous additivemanufacturing platform 10110, the user interface 10112, the data sources10114 and the design and simulation system 10116 referred to asdistributed manufacturing network entities constitute distributedcomputing nodes of the distributed ledger system.

The distributed computing node is essentially a computing device havinga processor and a computer-readable medium having machine-readableinstructions stored thereon and contains full copy of the transactionhistory of the distributed ledger. The nodes of the distributed ledgermay be implemented in a variety of computing systems including additivemanufacturing systems, enterprise systems, inventory management systems,packaging systems, shipping and/or delivery tracking systems, SKUdatabases, smart factories and so on. Whenever additional transactionsare proposed to be added to the distributed ledger, one or more of thenodes typically validate the proposed additional transaction records,such as via a consensus algorithm. Typically, once the proposedtransaction has been validated e.g., through any consensus algorithm,the proposed transaction is added to each copy of the distributed ledgeracross all the nodes.

In embodiments, the transaction data is validated by the nodes through aproof-of-work (POW) consensus algorithm and hashed into an ongoing chainof cryptographically approved blocks of transaction records constitutingthe distributed ledger.

In embodiments, proof of work algorithms require the nodes to perform aseries of calculations to solve a cryptographic puzzle. For instance, inorder to validate a pending data record, the nodes may be required tocalculate a hash via a hash algorithm (e.g., SHA256) that satisfiescertain conditions set by the system. The calculating of a hash in thismanner may be referred to herein as “mining,” and the nodes performingthe mining may be referred to as “miners” or “miner nodes.” Thedistributed ledger may, for example, require the value of the hash to beunder a specific threshold. In such embodiments, the nodes may combine a“base string” (i.e., a combination of various types of metadata within ablock header, e.g., root hashes, hashes of previous blocks, timestamps,etc.) with a “nonce” (e.g., a whole number value) to be input into thePOW algorithm to produce a hash. In an exemplary embodiment, the noncemay initially be set to 0 when calculating a hash value using the POWalgorithm. The nonce may then be incremented by a value of 1 and used tocalculate a new hash value as necessary until a node is able todetermine a nonce value that results in a hash value under a specifiedthreshold (e.g., a requirement that the resulting hash begins with aspecified number of zeros). The first node to identify a valid nonce maybroadcast the solution (in this example, the nonce value) to the othernodes of the distributed ledger for validation. Once the other nodeshave validated the “winning” node's solution, the pending transactionrecord may be appended to the last block in the distributed ledger. Insome cases, a divergence in distributed ledger copies may occur ifmultiple nodes calculate a valid solution in a short timeframe. In suchcases, the nodes using the POW algorithm accept the longest chain ofblocks (i.e., the chain with the greatest proof of work) as the “true”version of the distributed ledger. Subsequently, all nodes having adivergent version of the distributed ledger may reconcile their copiesof the ledger to match the true version as determined by the consensusalgorithm.

In other embodiments, the consensus algorithm may be a “proof of stake”(“PoS”) algorithm, in which the validation of pending transactionrecords depends on a user's “stake” within the distributed ledger. Forexample, the user's “stake” may depend on the user's stake in a digitalcurrency or point system (e.g., a cryptocurrency, token system, assetshare system, reputation point system, etc.) within the distributedledger. The next block in the distributed ledger may then be decided bythe pending transaction record that collects the greatest number ofvotes. A greater stake (e.g., in a given digital currency or tokensystem) results in a greater number of votes that the user may allocateto particular pending transaction records, which in turn increases thechance for a particular user to create blocks in the distributed ledger.In embodiments, a distributed ledger need not be based on a token orcryptocurrency system, but rather may be secured by conventional orother security techniques, for example. In embodiments, such as onesinvolving a digital thread, proof of stake may be weighted, such aswhere a product manufacturer's votes, a customer's votes, or the likecount more than an arbitrary third party.

In yet other embodiments, a consensus algorithm may be a “practicalbyzantine fault tolerance” (“PBFT”) algorithm, in which each nodevalidates pending transaction records by using a stored internal statewithin the node. In particular, a user or node may submit a request topost a pending transaction record to the distributed ledger. Each of thenodes in the distributed ledger may then run the PBFT algorithm usingthe pending transaction record and each node's internal state to come toa conclusion about the pending transaction record's validity. Uponreaching said conclusion, each node may submit a vote (e.g., “yes” or“no”) to the other nodes in the distributed ledger. A consensus isreached amongst the nodes by taking into account the total number ofvotes submitted by the nodes. Subsequently, once a threshold number ofnodes have voted “yes,” the pending transaction record is treated as“valid” and is thereafter appended to the distributed ledger across allof the nodes.

In embodiments, the nodes are paid a transaction fee for their miningactivities. In embodiments, the distributed ledger is a private andpermissioned blockchain controlled by a single entity or a consortium oftrusted entities, that's built using a pre-built API provided on CORDA,Hyperledger, and Quorum.

In embodiments, the distributed ledger is a public, permissionlessblockchain that's built on Ethereum or bitcoin blockchain. Inembodiments, the event data related to the movement of goods through thesupply chain in the trade finance network may be tracked using an IoTsubsystem.

In embodiments, transaction records stored in the distributed ledger maybe hashed, encrypted, or otherwise protected from unauthorized accessand may only be accessible utilizing a private key to decrypt the storedinformation/data.

The blockchain may be a single blockchain configured for storing alltransactions therein, or it may comprise a plurality of blockchains,wherein each blockchain is utilized to store transaction recordsindicative of a particular type of transaction. For example, a firstblockchain may be configured to store shipment data and supply chaintransactions, and a second blockchain may be configured to storefinancial transactions (e.g., via a virtual currency).

In embodiments, the distributed ledger system includes a decentralizedapplication downloadable by entities in the distributed manufacturingnetwork.

In embodiments, the distributed ledger system includes a user interfaceconfigured to provide a set of unified views of the workflows to the setof entities of a distributed manufacturing network.

In embodiments, the distributed ledger system includes a user interfaceconfigured to provide tracking and reporting on state and movement of aproduct from order through manufacture and assembly to final delivery tothe customer.

In embodiments, the distributed ledger system includes a system fordigital rights management of entities in the distributed manufacturingnetwork. In embodiments, the distributed ledger system stores digitalfingerprinting information of documents/files and other informationincluding creation, modification.

In embodiments, the distributed ledger system includes a cryptocurrencytoken to incentivize value creation and transfer value between entitiesin the distributed manufacturing network.

In embodiments, the distributed ledger system includes a system forattesting the experience of a manufacturing node.

In embodiments, the distributed ledger system includes a system forcapturing the end-to-end traceability of a part.

In embodiments, the distributed ledger system includes a system fortracking all transactions, modifications, quality checks andcertifications on the distributed ledger.

In embodiments, the distributed ledger system includes a system forvalidating capabilities of a manufacturing node.

In embodiments, the distributed ledger system includes smart contractsfor automating and managing the workflows in the distributedmanufacturing network.

In embodiments, the distributed ledger system includes a smart contractfor executing a purchase order covering the scope of work, quotation,timelines, and payment terms.

In embodiments, the distributed ledger system includes a smart contractfor processing of payment by a customer upon delivery of product.

In embodiments, the distributed ledger system includes a smart contractfor processing insurance claims for a defective product.

In embodiments, the distributed ledger system includes a smart contractfor processing warranty claims.

In embodiments, the distributed ledger system includes a smart contractfor automated execution and payment for maintenance.

FIG. 121 is a schematic illustrating an example implementation of adistributed manufacturing network where the digital thread data istokenized and stored in a distributed ledger so as to ensuretraceability of parts printed at one or more manufacturing nodes in thenetwork according to some embodiments of the present disclosure. A userof the distributed manufacturing network 10130 may provide the productrequirements in the form of a purchase order or a 3D printinginstruction set 10902. The 3D printing instruction set 10902 containskey specifications and requirements like product design, material forprinting, quantity to be printed, price that the user is willing to payfor the print and the timelines for completing the printing. The 3Dprinting instruction set 10902 may also include one or more files (e.g.,a CAD file and/or an STL file) and any accompanying instructions forprinting the product defined in the file.

Upon receipt, the 3D printing instruction set 10902 is tokenized andstored in the distributed ledger 10624 in the autonomous additivemanufacturing platform 10110. The underlying information in the 3Dprinting instruction set 10902 is stored in the form of a unique recordrepresented by a block number with an address on the distributed ledger,which in turn is represented by a cryptographic token. The cryptographictoken captures the value of the underlying information in the 3Dprinting instruction set 10902 as ownership or access rights to thedistributed ledger address and tracks the transfer of such ownershipbetween users of the distributed manufacturing network 10130. Forexample, in FIG. 121 , the 3D printing instruction set 10902 istokenized in the form of a random 256 bit integer A091BC3 . . . , andstored in the distributed ledger 10624 represented by address BC22. Asthe new block is added to the distributed ledger 10624 at node 10128 allthe copies stored at various nodes including the manufacturing node10100, the manufacturing node 10800 and the manufacturing node 10802 getupdated with the new block. The matching system 10632 in the autonomousadditive manufacturing platform 10110 may help with matching thepurchase order or the 3D printing instruction set 10902 with one or moremanufacturing nodes or 3D printers. The matching may be based on factorslike printer capabilities, locations of the customer and themanufacturing nodes, available capacity at each node, pricing andtimelines requirements. In embodiments, a smart contract operates on theledger, such as to trigger conditional logic embodied in the smartcontract, such as tracking satisfaction of delivery obligations,releasing insurance obligations (such as insurance covering productsduring shipment), and the like. In embodiments, the smart contract mayallocate financial value, such as to tax and customs authorities, tocredit and debit card issuers, to distributers and resellers, torecipients of commissions, to recipients of royalties, to recipients ofrebates, credits and the like, to shippers/carriers, and to themanufacturer, among others.

In embodiments, the matching system 10632 may determine that the parts10904 and 10910 of the product be matched to the manufacturing node10100 for printing, parts 10906 and 10908 to the manufacturing node10128 and parts 10912 and 10914 matched to the manufacturing node 10802.The assembly of all the parts into the final product may be matched tothe manufacturing node 10800.

Each of the part may also be tokenized to capture information includingpurchase order identifier (orderID), instruction set identifier(fileID), manufacturing node (manufacturerID), 3D printer (printerID),part number (partID) and part specifications containing information likematerial and quantity etc. and stored as a record or block in thedistributed ledger. The parts can then be tracked using a physicaltracker using a unique part number, engraving, RFID tags, bar codes orsmart labels linked to the block and unique to the token. In a similarmanner, the product assembled from all the parts may also be tokenizedand tracked as it moves through the distributed manufacturing network10130 and through various VCN entities 10126 to the customer.

In embodiments, tokenizing the part, product or 3D printed instructionset may include wrapping access, intellectual property, licensing,ownership, financial, time-sharing, leasing, rental, usage sharingand/or other suitable rights related to the part, product or instructionset into a token such that the access, licensing, ownership and/or othersuitable rights managed by one or more of the tokens.

In embodiments, the distributed manufacturing network 10130 may definepermissions and/or operations associated with the tokens. For example,the token may allow the tokenized 3D printed instruction set to beviewed, edited, copied, bought, sold, and/or licensed based onpermissions set at a time of tokenization by the distributedmanufacturing network 10130. In embodiments, the distributedmanufacturing network 10130 may provide for orchestration of adistributed manufacturing marketplace or exchange, such as where 3Dprinted instruction sets may be exchanged, such as, without limitation,through tokens that are optionally governed by smart contracts that maybe configured by a host of the distributed manufacturing exchange ormarketplace and/or by manufacturing nodes. For example, an exchange ormarketplace may host exchanges for tokenized 3D printed instructionsets, parts, products, expertise, trade secrets, insight, wheretransaction terms are pre-defined and/or configurable (such as withconfigurable smart contracts that enable various transaction models,including bid/ask models, auction models, donation models, reverseauction models, fixed price models, variable price models, contingentpricing models and others), where metadata is collected and/orrepresented about categories of distributed manufacturing marketplace orexchange, and where relevant content is presented, including marketpricing data, substantive content about additive manufacturing, contentabout providers, and the like. Such an exchange may facilitatemonetization of tokenized 3D printed instruction set knowledgerepresented in tokens.

In embodiments, a distributed manufacturing marketplace as describedherein, may be integrated with or within another exchange, such as adomain-specific exchange, a geography-specific exchange, or the like,where the distributed manufacturing marketplace may be configured toaddress the subject matter of the other exchange, such as: to accountfor changes in the other exchange in the models and algorithms used inthe distributed manufacturing marketplace (e.g., pricing models,predictive models, control systems, and others) to the extent that theyimpact, supply, demand, pricing, volumes, operational factors, and otherfactors; to provide via distributed manufacturing units a set of itemsand/or a set of data that may be used by the other exchange (such as byproviding products that can be exchanged in the other exchange, byproviding data sets, analytic measures, or the like that may inform theoperation of the other exchange and the like); to provide for resourcesharing between the distributed manufacturing marketplace and the otherexchange (such as to enable shared computation, shared data storage,shared network resources, shared security resources, shared physicallocation, and the like); and/or to provide for integrated coordinationof the distributed manufacturing marketplace and the other exchange.Shared resource utilization may include embedding a set of services ofthe other exchange in one or more additive manufacturing units, such asto render it a hybrid of an additive manufacturing unit and a unitenabling another exchange. The other exchange may be a product exchange(such as an e-commerce marketplace, an auction marketplace, or thelike), a stock exchange, a commodities exchange, a derivatives exchange,a futures exchange, an advertising exchange, an energy exchange, arenewable energy credits exchange, a knowledge exchange, acryptocurrency exchange, a bonds exchange, a currency exchange, aprecious metals exchange, a petroleum exchange, an exchange for goods,an exchange for services, an exchange for legal rights (such asintellectual property, real property, likeness, publicity rights,privacy rights, or others), or any of a wide variety of others. This mayinclude integration by APIs, connectors, ports, brokers, and otherinterfaces, as well as integration by extraction, transformation andloading (ETL) technologies, smart contracts, wrappers, containers, orother capabilities.

In embodiments, the digital twin system 10214 may be configured topresent a simulation of a marketplace, an exchange, a product, a seller,a buyer, a transaction, or a combination thereof via a marketplacedigital twin. The digital twin or replica may be a two-dimensional orthree-dimensional simulation of a marketplace, an exchange, a product, aseller, a buyer, a transaction, and the like. The digital twin may beviewable on a computer monitor, a television screen, a three-dimensionaldisplay, a virtual-reality display and/or headset, an augmented realitydisplay such as AR goggles or glasses, and the like. The digital twinmay be configured to be manipulated by one or more users of theautonomous additive manufacturing platform 10110. Manipulation by a usermay allow the user to view one or more portions of the digital twin ingreater or lesser detail. In embodiments, the digital twin system 10214may be configured such that the digital twin may simulate one or morepotential future states of a marketplace, an exchange, a product, aseller, a buyer, a transaction, etc. The digital twin may simulate theone or more potential future states of a marketplace, an exchange, aproduct, a seller, a buyer, a transaction, etc. based on simulationparameters provided by the user. Examples of simulation parametersinclude a progression of a period of time, potential actions by partiessuch as buyers or sellers, increases in supply and/or demand ofproducts, resources, etc., changes in government regulations, and anyother suitable parameters.

In embodiments, the autonomous additive manufacturing platform 10110 mayimplement gamification in the distributed manufacturing network 10130 byawarding points to various entities for performing tasks desirable tooperation of the distributed manufacturing network 10130. For example,points may be awarded for trading parts or products of a particular typeand/or within a particular region. Entities who have been awarded pointsmay compete with one another, and digital and/or physical prized may beawarded to entities who have achieved one or more point thresholdsand/or have ranked above one or more other entities on a pointsleaderboard.

In embodiments, the scoring system 10634 can rate the one or moremanufacturing nodes or 3D printers in the distributed manufacturingnetwork 10130 based on a customer satisfaction score for meetingcustomer requirements. In embodiments, the score may form another basisfor matching customers to manufacturing nodes or 3D printers.

In embodiments, the scoring system 10634 crowdsources the customersatisfaction score from multiple entities in the distributedmanufacturing network 10130. Examples of crowd sources includecertifying entities, domain experts, customers, manufacturers,wholesalers, and any other suitable party.

In embodiments, certifying entities or domain experts may certify one ormore 3D printed parts as being good quality, accurate, and/or reliable.In embodiments, customers may review and certify one or more 3D printedparts or products, such as to indicate that the part or product is inworking order and/or of expected quality. In embodiments, manufacturersand/or wholesalers may sign an instance of 3D printed instruction set,such as by applying a serial number to a piece of 3D printed instructionset before it is transmittable to a customer. Certifications, reviews,signatures, and/or any other validation indicia made by crowd sourcesmay be recorded in the distributed ledger, such as by adding one or morenew blocks to the distributed ledger that indicate the certification,review, signature, or other validation indicia.

In embodiments, the autonomous additive manufacturing platform 10110utilizes a system for learning on a training set of outcomes,parameters, and data collected from data sources associated with thedistributed manufacturing network 10130 to train models in theartificial intelligence system 10212 to predict and manage productdemand from one or more customers of the distributed manufacturingnetwork 10130.

In embodiments, the autonomous additive manufacturing platform 10110utilizes a system for learning on a training set of outcomes,parameters, and data collected from data sources associated with thedistributed manufacturing network 10130 to train models in theartificial intelligence system 10212 to predict and manage materialsupply.

In embodiments, the autonomous additive manufacturing platform 10110utilizes a system for learning on a training set of outcomes,parameters, and data collected from data sources associated with thedistributed manufacturing network 10130 to train models in theartificial intelligence system 10212 to optimize production capacity fora distributed manufacturing network enabled by the autonomous additivemanufacturing platform.

In embodiments, the autonomous additive manufacturing platform 10110utilizes a system for learning on a training set of outcomes,parameters, and data collected from data sources associated with thedistributed manufacturing network 10130 to train models in theartificial intelligence system 10212 to schedule across multipleproduction processes, printers, manufacturing nodes, and to recalibrateschedules dynamically based on changes in real-time production andpriority data.

In embodiments, the autonomous additive manufacturing platform 10110 mayutilize a distributed ledger to manage a set of permission keys thatprovide access to one or more instances of the 3D printing instructionset 10902 and/or services associated with the distributed manufacturingnetwork 10130.

In embodiments, the distributed ledger provides provable access to the3D printing instruction set 10902, such as by one or more cryptographicproofs and/or techniques.

In embodiments, the distributed ledger may provide provable access tothe 3D printing instruction set 10902, by one or more zero-knowledgeproof techniques.

In embodiments, the autonomous additive manufacturing platform 10110 maymanage the distributed ledger to facilitate cooperation and/orcollaboration between two or more entities with regard to one or moreinstances of the 3D printing instruction set 10902.

In embodiments, a trusted authority (e.g., the autonomous additivemanufacturing platform 10110 or another suitable authority) may issueprivate key and public key pairs to each registered user of thedistributed manufacturing network 10130. The private key and public keypairs may be used to encrypt and decrypt data (e.g., messages, files,documents, etc.) and/or to perform operations with respect to thedistributed ledger.

In embodiments, the autonomous additive manufacturing platform 10110 oranother suitable authority may provide two or more levels of access tousers.

In embodiments, the autonomous additive manufacturing platform 10110 maydefine one or more classes of users, where each of the classes of usersis granted a respective level of access.

In embodiments, the autonomous additive manufacturing platform 10110 mayissue one or more access keys to one or more classes of users, where theone or more access keys each correspond to a respective level of access,thereby providing users of different levels of access via theirrespective issued access keys.

In embodiments, possession of certain access keys may be used todetermine a level of access to the distributed ledger. For example, afirst class of users may be granted full viewing access of a block,while a second class of users may be granted both viewing access ofblocks and an ability to verify and/or certify one or more instances oftransactions contained within a block, and while a third class of usersmay be granted viewing access of blocks, an ability to verify and/orcertify one or more instances of transactions contained within a block,and an ability to modify the one or more instances of transactionscontained within the block. In some embodiments, a class of users may beverified as being a legitimate user of the distributed ledger in one ormore roles and allowed related permissions with respect to thedistributed ledger and content stored therein.

In embodiments, the distributed manufacturing network 10130 mayestablish a whitelist of trusted parties and/or devices, a blacklist ofuntrusted parties and/or devices, or a combination thereof for managingaccess.

In embodiments, the additive manufacturing platform 10110 may beconfigured to create customized products for shoppers (i.e., customers)in or traveling to a retail environment. The customized products may beprinted at the retail environment by the additive manufacturing unit10102, thereby attracting customers to the retail environment. Thecustomized products may include one or both of ornamental designs andfunctional designs. The ornamental designs may be configured to have oneor more aesthetic elements that are customized according to a profile ofthe customer. The functional designs may be configured to have one ormore functional features that are customized according to a profile ofthe customer. For example, the additive manufacturing platform may usecustomer profile information such as location data and/or search data todetermine that a customer will be visiting the retail environment. Upondetermining that the customer will be visiting the retail environment,the additive manufacturing platform may use information indicative ofaesthetic and/or functional desires of the customer to design acustomized product for the customer. The additive manufacturing unit10102 may manufacture the customized product such that the customizedproduct may be purchased by the customer from the retail environment.The customized product may be a product customized to fit the physiologyof the customer. For example, the customized product may be a case for acellular phone designed to fit a hand of the customer based on datarelated to the shape and/or size of the hand of the customer.

In embodiments, the additive manufacturing platform 10110 may beconfigured to create product samples tailored to shoppers. The additivemanufacturing platform 10110 may use data from the customer profile todetermine one or more types of product samples that may appeal to thecustomer. The additive manufacturing unit 10102 may print the productsamples that appeal to the customer prior to and/or during visitation tothe retail environment by the customer. The product samples may include,for example, material samples, fabric samples, food samples, or anyother suitable type of product sample.

In embodiments, the additive manufacturing platform 10110 may beconfigured to use images, text, and/or videos related to the customer tobuild the customer profile. The images, text, and/or videos may besourced from one or more of web crawlers, social media feeds, publicdatabases, and the like.

In embodiments, the additive manufacturing platform 10110 may includethe AI system 10212 configured to perform AI and/or machine learningtasks related to functions of the additive manufacturing platform. TheAI system 10212 may be configured to at least partially design thecustomized products for shoppers. The AI system 10212 may use one ormore machine learned models 10213 to analyze the customer profile anddetermine one or more customized products or features thereof that wouldbe desirable to the customer. The AI system 10212 may use one or moremachine learned models 10213 to analyze sources of images, text, and/orvideos to build the customer profile. The machine learned models 10213may be configured to allow the AI system 10212 to determine types ofimages, text, and/or videos that are more or less valuable and/oreffective to build the customer profile. The AI system 10212 may use oneor more machine learned models 10213 to determine types of customdesigns that may be more or less desirable to the customer.

In embodiments, the additive manufacturing platform 10110 may beconfigured to produce out-of-stock and/or low-stock products on-site atthe retail environment. The platform may receive data related to amountsof stock of products of the retail environment. The platform maydetermine that one or more products are out of stock and/or may becomeout of stock. The AI system 10212 may be configured to determine the outof stock products. Upon determining that one or more products are out ofstock and/or may become out of stock, the platform may, by using theadditive manufacturing unit 10102, produce more of the products.

In embodiments, the additive manufacturing platform 10110 may beconfigured to produce infrastructure for the retail environment. Theinfrastructure may be new infrastructure and/or replacementinfrastructure. The infrastructure may be produced via the additivemanufacturing unit 10102. Examples of infrastructure include pallets,storage racks, display environments, signs, packages, tags, escalatorparts, elevator parts, and the like. The additive manufacturing platform10110 may be configured to automatically determine infrastructure needsof the retail environment. The AI system 10212 may be configured to usea machine learned model to determine and/or predict infrastructure needsof the retail environment.

In embodiments, the additive manufacturing platform may be configured tocreate customized products for shoppers (i.e., customers) in ortraveling to a retail environment. The customized products may beprinted at the retail environment by a 3D printing device, therebyattracting customers to the retail environment. The customized productsmay include one or both of ornamental designs and functional designs.The ornamental designs may be configured to have one or more aestheticelements that are customized according to a profile of the customer. Thefunctional designs may be configured to have one or more functionalfeatures that are customized according to a profile of the customer. Forexample, the additive manufacturing platform may use customer profileinformation such as location data and/or search data to determine that acustomer will be visiting the retail environment. Upon determining thatthe customer will be visiting the retail environment, the additivemanufacturing platform may use information indicative of aestheticand/or functional desires of the customer to design a customized productfor the customer. The 3D printing device may manufacture the customizedproduct such that the customized product may be purchased by thecustomer from the retail environment. The customized product may be aproduct customized to fit the physiology of the customer. For example,the customized product may be a case for a cellular phone designed tofit a hand of the customer based on data related to the shape and/orsize of the hand of the customer.

In embodiments, the additive manufacturing platform may be configured tocreate product samples tailored to shoppers. The additive manufacturingplatform may use data from the customer profile to determine one or moretypes of product samples that may appeal to the customer. The 3Dprinting device may print the product samples that appeal to thecustomer prior to and/or during visitation to the retail environment bythe customer. The product samples may include, for example, materialsamples, fabric samples, food samples, or any other suitable type ofproduct sample.

In embodiments, the additive manufacturing platform may be configured touse images, text, audio, and/or videos related to the customer to buildthe customer profile. The images, text, audio, and/or videos may besourced from one or more of web crawlers, social media feeds, publicdatabases, and the like.

In embodiments, the additive manufacturing platform may include an AIsystem configured to perform AI and/or machine learning tasks related tofunctions of the additive manufacturing platform. The AI system may beconfigured to at least partially design the customized products forshoppers. The AI system may use one or more machine learned models toanalyze the customer profile and determine one or more customizedproducts or features thereof that would be desirable to the customer.The AI system may use one or more machine learned models to analyzesources of images, text, and/or videos to build the customer profile.The machine learned models may be configured to allow the AI system todetermine types of images, text, and/or videos that are more or lessvaluable and/or effective to build the customer profile. The AI systemmay use one or more machine learned models to determine types of customdesigns that may be more or less desirable to the customer.

In embodiments, the additive manufacturing platform may be configured toproduce out-of-stock and/or low-stock products on-site at the retailenvironment. The platform may receive data related to amounts of stockof products of the retail environment. The platform may determine thatone or more products are out of stock and/or may become out of stock.The AI system may be configured to determine restocking needs. Upondetermining that one or more products are out of stock and/or may becomeout of stock, the platform may, by the 3D printing device, produce moreof the products.

In embodiments, the additive manufacturing platform may be configured toproduce infrastructure for the retail environment. The infrastructuremay be new infrastructure and/or replacement infrastructure. Theinfrastructure may be produced via the 3D printing device. Examples ofinfrastructure include pallets, storage racks, display environments,signs, packages, tags, escalator parts, elevator parts, and the like.The additive manufacturing platform may be configured to automaticallydetermine infrastructure needs of the retail environment. The AI systemmay be configured to use a machine learned model to determine and/orpredict infrastructure needs of the retail environment.

In embodiments, an additive manufacturing platform 10110, such as thatassociated with a value chain or other network, may be designed,prepared, configured and/or deployed to support the design, development,manufacture and distribution of health and medical devices, components,parts, equipment and the like. For example, in connection with a patientconsultation with a medical or health services provider, an additivemanufacturing unit may be designated to support the consultation, suchas a mobile additive manufacturing unit 10102 and/or a unit located insufficiently close proximity to the medical or health services providerto facilitate rapid delivery of medical and healthcare hard goods anddevices produced by the additive manufacturing unit 10102.

Based on the nature of the healthcare consultation (e.g., medicalspecialty and its corresponding devices, equipment and parts), theadditive manufacturing unit 10102 may be equipped with appropriatematerials, such as a combination of metal and/or plastic printingmaterials, or other printing materials, that are suitable to print arange of possible health and medical devices, components, parts,equipment and the like to support healthcare providers and theirpatients.

In embodiments, the platform 10110 may take inputs from or related to ahealthcare consultation, such as inputs indicating a needed medicaldevice or part (e.g., technical specifications, CAD designs, and thelike); inputs indicating patient-specific data (e.g., clinical criteria,measurements such as sizing, weight, height, girth, circumference, orthe like); and inputs provided by medical and health service providersor other third parties, such as device specifications, requirements, andthe like (e.g., limitations on device size, such as thickness,requirements related to load- or stress-bearing minimums, or some othercriterion).

In embodiments, the platform 10110 may process the inputs from aplurality of sources including, but not limited to, medical records(e.g., patient measurements, material allergies, use of other relatedmedical devices, and the like), device specification data (e.g.,manufacturing specifications from the party(ies) holding rights to thedevice, part or other object to be manufactured), patient-input data(e.g., aesthetic preferences such as color of the device),healthcare-provider-input data (e.g., medical office branding), or someother input. An artificial intelligence system (such as a roboticprocess automation system trained on a training set of expert medicaldevices or other data), to determine a recommended action, prototype,device, which in embodiments may involve production of a device and/or acomponent of a device. The additive manufacturing platform 10110 may, insome such embodiments, automatically determine (such as using anartificial intelligence system, such as robotic process automationtrained on an expert data set) whether a medical device is readilyavailable from a manufacturer (including a device that is currently instock and/or on order) and/or whether an additive manufacturing systemshould produce the device, such as to meet an immediate patient need, tosave costs, or the like. Similarly, the additive manufacturing platformmay, in some embodiments, using similar systems, automatically determinethat an element should be additively manufactured to facilitate repair,such as where a complementary component may be generated to replace aworn or absent element of a medical device.

In an example embodiment, an outpatient may visit an orthopedic officefor a healthcare consultation relating to a knee injury. Given theprobability that the patient will require some form of external kneesupport from a medical device, such as a brace, an attending physicianin advance of the healthcare consultation may access a user interface,dashboard or some other user portal to the additive manufacturingplatform to determine the availability of knee braces and other medicaldevices to be manufactured by the additive manufacturing platform (e.g.,to confirm that the additive manufacturing platform 10110 has availabledesigns, CAD renderings and/or other specifications that will enable itto produce the needed medical device). If the additive manufacturingplatform 10110 has such device specifications, the attending physician(or other personnel associated with the upcoming patient healthcareconsultation) may place would-be wanted device designs in a queue hold,reserve or some other means of recording potential interest in theirmanufacture. By having such recording, upon meeting with the patient,the attending physician (or other personnel associated with the upcomingpatient healthcare consultation) may be able to present device optionsto the patient to select from, using the user interface, dashboard orsome other user portal to the additive manufacturing platform. If aneeded medical device is not currently associated with the additivemanufacturing platform, this may cause the platform to automaticallysend out a request for corresponding device specifications, design andother data that are needed to manufacture the device, component or part.Once such corresponding device specifications, design and other data arelocated, an alert may be provided back to the attending physician (orother personnel associated with the upcoming patient healthcareconsultation) indicating that there are proposed products/devices forreview that appear to conform with the listed device requirements. Aspart of the review of each available specification, design or other datathat is needed to manufacture the device, contract terms relating tocosts, warranty and other considerations may be presented for review.Contract terms and contractual relationships between users of theadditive manufacturing platform and third party holders of rightsrelated to device manufacturing may be coordinated using smartcontracts, as described herein. Before, during, or after the patient'shealthcare consultation, a medical device design may be selected andinput for manufacture to the additive manufacturing platform. As part ofthe order, data relating to the specific patient may be submitted to theadditive manufacturing platform, such as data regarding thecircumference of the patients lower-leg, knee, and upper-leg that areneeded to make an appropriately sized brace. Such information may bemanually input to the additive manufacturing platform or may beautomatically input to the additive manufacturing platform by transferof data from a data source external to the additive manufacturingplatform 10110, such as an electronic medical record, or some other datasource storing data that is relevant to the device characteristics.Additional, preferential data may also be provided, such as a childwanting images of koala bears engraved in the exterior of their brace,or a businessperson wanting the brace to be a particular color to bettermatch her skin tone and/or business suit color, to make the brace lessapparent. The user interface, dashboard or some other user portal to theadditive manufacturing platform may enable interaction with the additivemanufacturing platform that allows a user, like a patient, to seedifferent prototypes and aesthetic flourishes of the to-be manufactureddevice, prior to submitting a job to be built. Upon finalizing thedesign specifications, the additive manufacturing platform 10110 mayproceed with producing the device and/or a component or part of thedevice, while the patient's healthcare consultation proceeds, or thismanufacture may be finalized following the consultation, and the deviceautomatically sent to the patient and/or healthcare provider based oncontact data input to the additive manufacturing platform 10110 at thetime of placing the order.

In embodiments, the additive manufacturing platform 10110, such as thatassociated with a value chain network may be prepared, configured,and/or deployed to support printing of customized and/or personalizedhotel textiles for a set of hotel guests. In one example, in connectionwith an upcoming hotel guest visit, the additive manufacturing unit10102 may be designated for support, such as a mobile additivemanufacturing unit 10102 and/or a unit located in sufficiently closeproximity to the hotel to facilitate rapid delivery of items produced bythe additive manufacturing unit 10102. In embodiments, textiles that maybe customized and/or personalized may include bedding, sheets, towels,robes, pillows, blankets, curtains, furniture, and the like.

In embodiments, the additive manufacturing unit 10102 may be equippedwith appropriate materials, such as a combination of fabrics and otherprinting materials, that are suitable to print a range of possibletextiles, or other elements to support the hotel visit. In embodiments,fabrics may include, but are not limited to, canvas, cashmere, chenille,chiffon, cotton, crepe, damask, georgette, gingham, jersey, lace,leather, linen, merino wool, modal, muslin, organza, polyester, satin,silk, spandex, suede, taffeta, toile, tweed, twill, velvet, viscose, andmany others.

In embodiments, the additive manufacturing platform 10110 may takeinputs related to the upcoming hotel visit, such as inputs indicatingthe type(s) of item to print (e.g., pillows, bedding, towels, and thelike); inputs indicating fabric type (such as cotton, silk, or thelike); inputs indicating item size (such as to fit a queen bed or kingbed); and inputs captured by cameras, microphones, data collectors,sensors, and other information sources associated with the upcominghotel visit. For example, a hotel employee may capture informationrelated to hotel guest preferences. In embodiments, the additivemanufacturing platform 10110 may process the inputs, such as using theartificial intelligence system 10212 (such as a robotic processautomation system trained on a training set of expert service visitdata), to determine a recommended action, which in embodiments mayinvolve printing of a textile. The additive manufacturing platform 10110may, in some such embodiments, automatically determine (such as using anartificial intelligence system 10212, such as robotic process automationtrained on an expert data set) whether the additive manufacturing unit10102 should produce the textile.

In any such embodiment that recommend or configure instructions foradditive manufacturing, the additive manufacturing platform 10110 maydiscover available materials/fabrics, configure instructions, andinitiate additive manufacturing, and provide updates to a hotelemployee, such as updates as to when an element will be ready to use.

In embodiments, the additive manufacturing platform 10110, such asthrough a trained AI agent, may automatically configure and schedule aset of jobs across a set of additive manufacturing units 10102 withawareness of the status of other relevant entities involved in otherworkflows, such as what other work is being done (e.g., to allow forappropriate sequencing of additive manufacturing outputs that align withoverall workflows), the priority of the printing job (e.g., whether itrelates to a loyal hotel guest), or other factors. In embodiments,optimization of workflows across a set of additive manufacturingentities may occur by having the artificial intelligence system 10212undertake a set of simulations, such as simulations involvingalternative scheduling sequences, design configurations, alternativeoutput types, and the like. In embodiments, simulations may includesequences involving additive manufacturing and other manufacturingentities (such as subtractive manufacturing entities that cut, dye, orthe like and/or finishing entities that sew, configure, add hotel guestinitials or the like), including handoffs between sets of differentmanufacturing entity types, such as where handoffs are handled byrobotic handling systems. In embodiments, a set of digital twins mayrepresent attributes and capabilities of the various manufacturingsystems, various handling systems (robotic systems, arms, conveyors, andthe like, as well as human workforce) and/or the surrounding environment(such as a hotel, a manufacturing facility, or the like).

In embodiments, the additive manufacturing platform 10110 such as thatassociated with a value chain network may be prepared, configured and/ordeployed to support restaurant operations. For example, in connectionwith a customer reservation at a restaurant, the additive manufacturingunit 10102 may be designated to support the customer reservation, suchas a table-side additive manufacturing unit 10102 and/or a portable unitto facilitate direct-to-table delivery of items produced by the additivemanufacturing unit 10102.

Based on the nature of the reservation (e.g., special dietaryrequirements, accessibility requirements, occasion of the reservation)and the services and supplies available at the restaurant, the additivemanufacturing unit 10102 may be equipped with appropriate materials,such as a combination of food grade service/storage materials and otherprinting materials, that are suitable to print a range of possibleservice items, specialized flatware, customizedcommemorative/celebration items, or other elements to support thereservation. In embodiments, the additive manufacturing platform 10110may take inputs from or related to the reservation, such as inputsindicating time of day, size of the party, special requests, affiliationwith principals of the restaurant, loyalty participation, and the like;inputs indicating service support capabilities at the restaurant andoptions for timely access to locally available service supportmaterial/equipment (such as a status of ovens, cook tops, food storage,meal prep material, customizable service items, or the like); and inputscaptured by cameras, microphones, data collectors, sensors, and otherinformation sources associated with the reservation, including selectinput capture device(s) associated with one or more participants in thereservation (e.g., a personal mobile phone with image capture features).For example, a hostess station camera may capture a set of photos of theparticipants, such as images of the reservation participant(s) facesthat are suitable for generation of a 3D data set for additivemanufacturing printing use.

In embodiments, the additive manufacturing platform 10110 may processthe inputs, such as by using the artificial intelligence system 10212,to determine a recommended action for servicing participants in thereservation, which in embodiments may involve use of a service item,such as an standard service item adapted to meet a service requirementof the reservation, such as a customized serving tray with separatedcompartments for each participant in the reservation, an item offlatware and/or serving spoon adapted for use by a person without anormal appendage, and the like. The additive manufacturing platform10110 may, in some such embodiments, automatically determine, such as byusing the artificial intelligence system 10212, trained on an expertdata set, and the like whether a suitable service item is readilyavailable and/or whether use of an additive manufacturing system toproduce the service item(s) can reduce delay, to save costs, or thelike. Similarly, the additive manufacturing platform 10110 may, in someembodiments, using similar systems, automatically determine that anelement should be additively manufactured to facilitate use ofadditional kitchen equipment, such as cook tops to ensure timely mealservice for the reservation, such as where a complementary component maybe generated to replace a worn or absent component, such as a gassetting knob on a gas range regulator.

In embodiments, automatic determination may occur using a machine visionsystem that captures a set of facial images of reservation participantsand produces an instruction set for additively manufacturing acomplementary service item, such as a drinking glass that matches thefacial image. In any such embodiment that recommends or configuresinstructions for additive manufacturing, the additive manufacturingplatform 10110 may discover available additive manufacturing units 10102(e.g., a drinking glass additive manufacturing unit on the restaurantpremises), configure compatible instructions, initiate additivemanufacturing, and provide updates to the service staff, such as updatesas to when the custom printed drinking glass will be ready to use. Inembodiments, the additive manufacturing platform 10110, such as througha trained AI agent, may automatically configure and schedule a set ofjobs across a set of additive manufacturing units 10102 (drinking glassadditive manufacturing units, kitchen equipment parts additivemanufacturing units, takeaway/takeout food storage systems additivemanufacturing units, and the like) with awareness of the status of otherrelevant reservations at the restaurant and other kitchens/serviceworkflows, such as the timing of food preparation/meal courses (e.g., toallow de-prioritization of additive manufacturing jobs that are toproduce reservation-related service items that won't be used immediatelyupon the start of the reservation), what other additive manufacturingwork is being done for other reservations (e.g., to allow forappropriate sequencing of additive manufacturing outputs that align withoverall kitchen workflows, meal service, and the like), the cost (bothdirect and indirect) of delays in additive manufacturing element access(e.g., poor reviews, discounted charges, lower service tip, freefood/beverage items as compensation for delays, and the like), or otherfactors.

In embodiments, restaurant service items that may be enhanced and/orproduce through additive manufacturing techniques include, withoutlimitation takeout/away containers constructed to meet individual fooditem needs, such as keeping salad cool, keeping a hot meal warm, keepinga serving of French fries crispy, containers shaped to meet food serviceitem size/shape (e.g., a triangle sized container for a slice of pie,round for a pancake, oblong/square for a sandwich item) and the like. Inembodiments, user-specific flatware, such as age range-specific flatwaresuitable for use by a baby just learning to use a fork and spoon or achild honing her skill with a knife, an unconventional flatware itembased on user preferences (explicitly expressed in association with thereservation) or (implicitly derived from user context/imagery) and thelike. Further in embodiments, table and service items, such as mugs,coasters, chargers, plates, and the like may be produced to meetreservation aspects, such as a logo supplied with the reservation, anoccasion-specific design/embellishment recommended during thereservation process, and the like. In embodiments, optimization ofworkflows across a set of additive manufacturing entities/units mayoccur by having an artificial intelligence system undertake a set ofsimulations, such as simulations involving alternative food preparationand/or reservation sequences, design configurations, alternativeoutput/material types, and the like.

In embodiments, reservation service items that rely on a mix of additivemanufacturing materials, such as paper-like material and thermalinsulation structures may provide performance benefits oversingle-material items, such as lower thermal transfer from an interiorof a service item (e.g., a custom printed drinking glass) to an exteriorof the item (e.g., for maintaining the interior temperature andimproving comfort of a user holding the glass).

In embodiments, the additive manufacturing platform 10110, such as thatassociated with a value chain network may be prepared, configured and/ordeployed to support printing of personalized food at campuses inuniversities and/or enterprises. In one example, an additivemanufacturing unit 10102 may be designated to provide ethnic andpersonalized food to students and workers on the go. In embodiments, theadditive manufacturing unit 10102 may be equipped with materials, suchas a combination of ingredients and other printing materials, that aresuitable to print a range of possible food items to support the studentsor workers. For example, pizza making may be automated by the additivemanufacturing unit 10102 and a multi-nozzle print head may depositdough, sauce and cheese along with personalized choice of pizzatoppings. Similarly, desserts, chocolates, cakes, pastries, even edibleplates, utensils and cutlery and the like may be printed by the additivemanufacturing unit 10102.

In embodiments, the additive manufacturing platform 10110 may takeinputs from or related to the customer, such as inputs indicating thetype(s) of food items to print (e.g., pizza, pasta, desserts, and thelike); inputs indicating taste preferences (such as spicy, sweet, or thelike); inputs indicating aesthetic preferences (such as texture, color,or the like); inputs indicating food item size (such as small, medium orlarge); inputs indicating nutritional requirements (proteins,carbohydrates, fats, vitamins, minerals etc.) inputs indicating healthneeds (such as allergies, or the like), and inputs captured by cameras,microphones, data collectors, sensors, and other information sourcesassociated with the upcoming campus visit, or some other input type. Forexample, information related to customer biological information may becaptured to determine that the customer does not have any seafoodallergies. In embodiments, the additive manufacturing platform 10110 mayprocess the inputs, such as using the artificial intelligence system10212 (such as a robotic process automation system trained on a trainingset of expert service visit data), to determine a recommended action,which, in embodiments, may involve printing of, for example, a customsushi that optimizes ingredients that fulfill the nutritionalrequirements of the customer.

In embodiments, the additive manufacturing unit 10102 may print takeoutcontainers to meet individual food item needs, such as keeping saladcool, keeping a hot meal warm, keeping a serving of French fries crispy,containers shaped to meet food service item size/shape and the like.

In embodiments, the food items may be printed at a mobile additivemanufacturing unit 10102 near or at the point of use on an on-demandbasis thereby reducing food inventory and the cost involved with storageand transportation.

In embodiments, the additive manufacturing platform 10110, such asthrough a trained AI agent, may automatically configure and schedule aset of jobs across a set of additive manufacturing units 10102 (e.g.,units creating food, desserts, plates, utensils, cutlery, kitchenequipment and the like) with awareness of the status of other relevantentities involved in other workflows, such as what other work is beingdone (e.g., to allow for appropriate sequencing of additivemanufacturing outputs that align with overall workflows), the priorityof the printing job (e.g., based on the timing of a customer order), orother factors. In embodiments, optimization of workflows across a set ofadditive manufacturing entities may occur by having an artificialintelligence system undertake a set of simulations, such as simulationsinvolving alternative scheduling sequences, design configurations,alternative output types, and the like. In embodiments, simulations mayinclude sequences involving additive manufacturing and othermanufacturing entities (such as subtractive manufacturing entities thatcut, drill, or the like and/or finishing entities (that decorate, plate,garnish, arrange, glaze or the like), including handoffs between sets ofdifferent manufacturing entity types, such as where handoffs are handledby robotic handling systems.

In embodiments, the additive manufacturing platform 10110 may beconfigured as a fixed or mobile system that operates individually or aspart of a network, to combine live inputs, library data, personal data,licensed data, and so forth to autonomously design and produce uniqueparts associated with a live event, for example, personalized mementos,sample products, limited edition artwork, and the like.

In embodiments, the additive manufacturing platform 10110 may acquirereal-time or personalized input from the user or venue using 3D scanningsuch as laser or white light scanners, image recognition, photography,publicly available data, etc. and combine and process the informationwith existing public or licensed part and data libraries to produce acombined 3D printable dataset and finished products that may bedelivered as the customer waits, or at a later time to a home, business,or venue seat.

In embodiments, the additive manufacturing platform 10110 such as thatassociated with a value chain network may be configured and deployed byfirst responders to support first responder events. For example, inconnection with a first responder request, the additive manufacturingunits 10102 may be designated to support design and print customcomponents, parts, equipment, medical devices, accessories and the likeon an on-demand real time basis. Some examples of equipment that may beprinted include Personal Protective Equipment (PPE), face shields,goggles or medical glasses, protective eyewear, boots, surgical hoods,earplugs, valves, nozzles, helmets, body shields, extrication tools andthe like.

In embodiments, the equipment may be printed near or at the point of useon a need basis. For example, eyewear, earplugs, helmets, boots may becustom printed based on the patient measurement. Similarly, equipmentincluding respirators, ventilators, custom valves and nozzles may beprinted at a mobile additive manufacturing platform based on immediatepatient needs and delivered at the point of care.

In embodiments, the additive manufacturing platform 10110 mayautomatically determine (such as using the artificial intelligencesystem 10212 trained on an expert data set) that one or more partsshould be additively manufactured to facilitate repair, such as where acomplementary part may be generated to replace a worn or absent elementof a first responder equipment or device. The additive manufacturingplatform 10110 may then process the inputs, such as by using theartificial intelligence system 10212, to determine a recommended actionfor servicing the repair request.

In embodiments, a set of additive manufacturing units 10102 may beprovided as shared resources for multiple tenants of a building, such asa commercial real estate building, where the additive manufacturingunits 10102 are integrated with other building resources, such asnetworking resources (e.g., RF, cellular, Wifi, fiber optic and otherresources), computational resources (e.g., data storage resources, edgeand cloud computational resources), IoT resources (e.g., cameras,sensors, and the like) and others, such that the capabilities of theadditive manufacturing units 10102 may be accessed by tenants accordingto terms and conditions of a lease (which in embodiments may beembodied, at least in part, as a smart contract that operates on datafrom or about the additive manufacturing units 10102). In embodiments,the additive manufacturing platform 10110 may include, link to, orintegrate with a set of devices, systems, services and other resourcesin a backbone for building, campus, or the like, including a set ofnetwork backbone and/or connectivity resources (such as 5G and othercellular network devices and infrastructure, such as switches, accesspoints, gateways, routers, wireless mesh network systems, satellitesystems, Wifi systems, long-range RF systems (such as LORA), Zigbee,Bluetooth and other wireless systems, as well as fixed network systems,such as fiber access gateways and other systems, modems and othergateway devices for cable, ethernet, digital subscriber line, analogtelephone line and other wired networking systems, each using any of awide range of protocols, such as ethernet, TCP/IP, UDP, and manyothers). Shared connectivity resources may include resources forInternet connectivity (such as wireless internet service provider (WISP)resources and fixed ISP connectivity), cellular connectivity (e.g.,shared 5G), mesh network connectivity, and many others. In embodiments,the additive manufacturing platform 10110 may include, link to, orintegrate with a set of shared data storage resources, such as ablockchain dedicated to the building, campus, or the like, a distributedledger, a database or other data repository, a distributed memory systemusing memory of devices and systems that provide the building's ITinfrastructure, and others. In embodiments, the additive manufacturingunits 10102 and other shared resources may be provisioned, such as by ahost or a trained intelligent agent operating on behalf of the host, toenable rapid customization and fulfillment of needs of tenants, such astenants of a building, campus, city, or the like, including operationalneeds (such as for spare parts, products, tools, accessories, supplies,replacement parts, and the like, among many others) and many others.Among many examples, additive manufacturing units 10102 may produceelements needed for specialized tenants, such as personal protectiveequipment, ventilators, wearable items, tools, or the like, as well aselements needed for IT infrastructure (such as connectors, plugs and thelike, such as to fiber optic cables, Ethernet ports, and the like), andmany others. In embodiments, the shared resources may be monitored, suchas with various utilization tracking techniques, such as event logs ofnetworking nodes, logs of software systems, and the like, and may beprovisioned by an automated provisioning system, including allocatingpayment responsibilities, allocating usage rights, settingprioritization of resource utilization (such as by tenant, by time, bytask, and the like). This may include automated management by anartificial intelligence agent that is trained by a training set of dataof expert resource managers. This may be a supervised, semi-supervisedor deep learning process, and may include training on outcomes, such asprofitability outcomes, tenant feedback outcomes, user satisfactionoutcomes, security outcomes, operational outcomes, and many others.Resource sharing and payments may be governed and controlled by a smartcontract, such as with governing rules for allocating resources andconditional logic determining prioritization and/or paymentresponsibilities, optionally operating on a distributed ledger of eventsinvolving the resources. In embodiments, the smart contract frameworkmay itself be a shared resource offered to tenants, such as to enablethem to offer services, share resources (such as with other tenants,including any of the resources noted herein as well as others), and thelike.

Liquid Lens

FIGS. 122-127 relate to various embodiments and applications of liquidlens devices. Liquid lens devices may be used in an assortment ofapplications, including for autonomous systems that rely on imageclassification to perform tasks. Liquid lens devices may be integratedinto many different areas of a value chain to improve performance ofvarious autonomous systems by providing improved image sensingcapabilities and image classification, amongst other things.

FIG. 122 is a diagrammatic view illustrating an example implementationof a conventional computer vision system 11100 for recognizing an object11102 of interest. The computer vision system 11100 includes a lensassembly 11104 that attempts to focus light from the object 11102 onto asensor 11106. The sensor 11106 may be an image sensor such as a chargecoupled device (CCD) or complementary metal oxide semiconductor (CMOS)device containing array of photo sensitive elements. The sensor mayconvert the light into analog electrical signal corresponding to lightintensity. An analog to digital (AD) converter 11108 then convertsanalog voltage into digital data. This raw digital data is then sent toan image processing system 11110 for analysis. The image processingsystem 11110 then processes the raw digital data to generate an image11112. The image processing system 11110 may also involve pre-processingand post-processing including image scaling, noise reduction, coloradjustment, brightness adjustment, white balance adjustment, sharpness,adjustment, contrast adjustment and the like to enhance the imagequality. Further the image may be analyzed using machine learning orother algorithms to identify one or more objects in the image.

Conventional computer vision systems 11100 have many limitations. Theattempt to recreate vision by creating focused images leads to the lossof a large amount of information and leaves the vision system 11100 withlimited data. The computer vision system 11100 typically generate twodimensional images of three-dimensional objects and are unable tocapture information related to aspects like object depth, motion,orientation and the like. The algorithms in the computer vision system11100 attempt to infer information about a 3D scene/object from 2Dframes and information thereby limiting the quality of inferences.

FIG. 123 is a schematic illustrating an example implementation of adynamic vision system 11200 for dynamically learning an object conceptabout an object 11202 of interest according to an embodiment of thepresent disclosure. The dynamic vision system 11200 may replace and/oraugment the lens 11104 of a conventional vision system 11100 with avariable focus liquid lens 11204. The variable focus liquid lens 11204may be an electrically controlled cell containing optical-grade liquid,that is deformed through electric current, changing the shape of thelens. The dynamic vision system 11200 leverages this flexibility ofliquid lens 11204 by constantly adjusting lens parameters to dynamicallychange various optical characteristics of light that pass through thelens including focal length, spherical aberration, field curvature,coma, chromatics aberrations, distortion, vignetting, ghosting andflaring, and diffraction of light. A fully variable liquid lens thusallows for more dynamic input for a sensor 11206 enabling it to capturevisual information and metadata that is otherwise lost in theconventional computer vision system 11100.

An analog to digital (AD) converter 11208 may generate digital data fromthe rich visual information captured at the sensor 11206 and an imageprocessing system 11208 with pre-processing, and post-processingcapabilities may generate images that are based with additional opticalparameters as part of image. The processing system 11209 may alsoinclude a control system 11212 configured to adjust one or more opticalparameters in real time including focal length, liquid materials,specularity, color, environment and lens shape. An adaptive intelligencesystem 11214 may then dynamically learn on a training set of outcomes,parameters, and data collected from the liquid lens 11204 to generate anobject concept 11216. The object concept 11216 may include contextualintelligence about the object and its environment which may then beprocessed by adaptive intelligence system 11214 to recognize the object11202.

In embodiments, the adaptive intelligence system 11214 may includeartificial intelligence capability, such as involving machine learningor other algorithms, neural networks, expert systems, models and others,to process the input data from the liquid lens and dynamically learn theobject concept to provide superior object recognition and vision.

In embodiments, adaptive intelligence system 11214 may be implemented asthe intelligence layer 140 that receives requests from a set ofintelligence layer clients and responds to such request by providingintelligence services to such clients (e.g., a decision, aclassification, a prediction or the like).

In embodiments, the dynamic vision system 11200 may feed real-timeadjustable data streams to the processing system 11209 to generatesituational awareness or create out-of-focus images of the object 11202so as to capture large amounts of information that is otherwise lostwhen inferring depth and distance in a focused image of a conventionalvision system 11100. The dynamic input to the liquid lens 11204 mayprovide richer metadata for image processing as the images are based onadditional optical parameters than just focal length and aperture. Theimage processing system 11210 may incorporate previously lostinformation so as to generate new set of insights about the object andits surroundings not captured by the conventional computer visionsystems 11100.

Compared to conventional computer vision systems 11100, that utilizefixed sensory elements, the dynamic vision system 11200 provided hereinmay utilize a dynamically learned liquid lens assembly. The conformableliquid lens 11204 in the assembly may be continuously, and/orfrequently, adjusting based on, for example, environment factors and/oron feedback from the processing system 11209 to generate training datathat is deeper in context and that corresponds to the physical lightthat the image represents. By training the dynamic vision system 11200to recognize objects using variable optical parameters through theliquid lens assembly, the processing system 11209 may learn an optimumoptical setting(s) to detect an object. The more dynamic input to thedynamic vision system 11200 may result in creating a richer context andproviding superior object recognition.

The dynamic vision system 11200 may integrate sensing, control andprocessing functions and dynamically adjusts the liquid lens 11204 asthe vision algorithms in the processing system 11209 take differentinputs to produce a real-world vision result.

The dynamic vision system 11200 mimics biological vision by integratingsensing, control and processing functions (biological vision involves astream of information passing directly through deep learning systemswhere these deep learning systems can directly change aspects of visionprocessing, including orientation, fovea centralis attention, eyelidactions, blinking and communication with other humans).

In embodiments, the dynamic vision system 11200 may utilize saccades tocharacterize objects by context and build a rich model of the object inits environment by capturing contextual intelligence throughassociations. This mirrors how saccades capture information about anobject in its environment. Saccade denotes a quick, simultaneousmovement of both eyes between two or more areas of focus. While viewinga scene, human eyes make sporadic saccadic movements stopping severaltimes while locating key parts of the scene, moving quickly between eachstop and building up a mental three-dimensional map corresponding to thescene. The dynamic vision system 11200 and methods described herein mayuse saccades to characterize objects by context and allow control of anoptical system to more quickly identify and characterize a field ofview. Saccades integrate varying physical/optical properties, along withobject-oriented learning, to rapidly improve understanding and search inthe visual sphere.

In embodiments, the dynamic vision system 11200 may also mimicbiofeedback loops of human babies to create a system of associativememory and vision and build a causal three-dimensional model of theenvironment. The learning system in human babies involves many feedbackloops of activities wherein babies build a causal model of the worldaround them by performing sequences of controlled experiments. Thedynamic vision offered by the liquid lens-based vision system may, inpart, mirror the learning algorithm of babies by starting a training setaround the object and letting its learning algorithm figure out theright way to look at the object.

FIG. 124 depicts a schematic illustrating an example architecture of adynamic vision system 11300 depicting a detailed view of variouscomponents according to some embodiments of the present disclosure. Thedynamic vision system 11300 for recognizing an object 11302 may includean optical assembly 11304 and a processing system 11306. The opticalassembly 11304 may include a conformable liquid lens 11308, a sensor11310 and an analog to digital (AD) converter 11312. The processingsystem 11306 may include a control system 11314, an image processingsystem 11316, an adaptive intelligence system 11318, a digital twinsystem 11320 and a simulation system 11322. The adaptive intelligencesystem may include a machine learning system 11324 and an artificialintelligence system 11326.

The conformable liquid lens 11308 of the optical assembly 11304 mayfrequently adjust in real-time based, in part, on change in one or moreoptical parameters by the control system 11314 creating real-time datastreams at the sensor 11310 which are then provided to the processingsystem 11306 to generate a situational awareness or computerizedunderstanding of the world that the dynamic vision system 11300 isoperating in. This understanding may include rich contextualintelligence about the object and its environment and may be representedas an object concept. The object concept may be used by the processingsystem for object recognition, predicting object motion, location andorientation, creating a 3D model of the object, monitoring the objectfor any defects and other applications. For example, the adaptiveintelligence system 11318 may process the object concept to build athree-dimensional representation of the object. The machine learningsystem 11322 in the adaptive intelligence system 11318 may input theobject concept into one or more machine learning models, the objectconcept being used as training data for the machine learning models.Further, the artificial intelligence system 11326 may be configured tomake classifications, predictions, and other decisions relating to theobject including determining the position, orientation and motion of theobject.

In embodiments, the dynamic vision system 11300 may be configured toprocess sensor information to create a three-dimensional representationof the object 11302 in a single step without the intermediate step ofprocessing into flat images.

In embodiments, the control system 11314 may provide controlinstructions to one or more actuators which in turn drive theadjustments in liquid lens configurations. The actuators may be operatedby a source of energy, typically electric current, hydraulic fluidpressure, or pneumatic pressure, and convert that energy into motion.Examples of actuators may include linear actuators, solenoids, combdrives, digital micromirror devices, electric motors, electroactivepolymers, hydraulic cylinders, piezoelectric actuators, pneumaticactuators, servomechanisms, servo motors, thermal bimorphs, screw jacks,or any other type of hydraulic, pneumatic, electric, mechanical,thermal, magnetic type of actuator, or some other type of actuator.

In embodiments, the control system 11314 may provide controlinstructions to one or more actuators to change the focal length of theliquid lens based on stimulation. This may provide the dynamic visionsystem 11300 with an auto-focus capability by focusing, refocusing ordefocusing the lens to a desired focal length. The stimulation mechanismmay include electrical, hydraulic, pneumatic, mechanical, thermal ormagnetic.

Some examples of control systems 11314 include electrowetting, soundpiezoelectrics and electro-active polymers.

In embodiments, the conformable liquid lens assembly in the dynamicvision system 11300 may have an electrowetting control system such thatan application of electrical voltage to the fluid in the liquid lenschanges the shape of the liquid, effectively changing the focus of theliquid lens assembly.

In embodiments, the placement of actuators in a variable focused liquidlens based optical assembly may be optimized using machine learning.

In embodiments, the control system 11314 may control the liquid lens11304 configuration based on feedback from the processing system 11306in response to a change in environment factors. Some examples of theenvironmental factors include temperature, vibrations, ambient sensordata, workflows, entity IDs, user behavioral data, entity profiling,similarity to known data and the like.

In embodiments, the control system 11314 may control the liquid lens11304 configuration based on feedback from the processing system 11306in response to a change in source lighting including control color,color temperature, timing (PWM), amplitude (e.g., increase PWM butdiminish amplitude, direction, polarization, and the like.

In embodiments, the control system 11314 may control the liquid lensconfiguration based on human occupancy and awareness of when lightingneeds to be coordinated with human needs versus adjusted solely to servethe liquid lens system.

In embodiments, the optical assembly 11304, may include multiple sets ofliquid lenses with the processing system 11306 coordinating the controlof multiple liquid lenses setup.

In embodiments, the optical assembly 11304, may include multiple sets ofliquid lenses with each lens having a separate objective function, and aseparate processing system with AI setups or algorithms.

In embodiments, the optical assembly 11304, may include one or moreliquid lens combined with a conventional convex or concave optical lenswith the processing system 11306 coordinating the control of thecombination.

In embodiments, the processing system 11306, such as using the adaptiveintelligence system 11318, the digital twin system 11320 and thesimulation system 11322 may execute simulations to model, simulate andcharacterize the mechanical, optical, or lighting aspects of the dynamicvision system 11300. The simulations executed by the processing system11306 may help identify suitable imaging components for the dynamicvision system 11300 including sensors, lenses and lights. Thesimulations may include real time analytics to calculate wide range ofmetrics, build charts, graphs and models and visualize the effect ofchange of one or more optical parameters on the performance of thedynamic vision system 11300. The artificial intelligence system 11326 inthe adaptive intelligence system 11318 may then utilize the one or moremodels to make classifications, predictions, recommendations, and/or togenerate or facilitate decisions or instructions relating to the lensmaterials, geometry, optical properties, performance and design of thedynamic vision system 11300. For example, the artificial intelligencesystem 11326 may execute simulations on one or more liquid lens digitaltwins for generating recommendations relating to the fluid used in theliquid lens. The simulations may be performed using different fluidsincluding distilled water, methyl alcohol, ethyl alcohol, ether, carbontetrachloride, methyl acetate, glycerine, nitrobenzene and the like togenerate recommendations on the preferred fluid for a given applicationof the dynamic vision system 11300.

The dynamic vision system 11300 may utilize dynamically learned sensoryelements to recognize objects ensuring a richer object recognitioncapacity that may be applied to a very wide range of use cases. Theapproach is ideal for imaging applications requiring rapid focusing,high throughput, and depth of field and working distance accommodation.Moreover, the approach is especially beneficial for complex visionapplications where conventional vision technologies have beeninadequate. Some examples of such applications include: recognizingobjects in dynamic environments like when the object or vision systemare moving; recognizing three dimensional (3D) objects by capturingdepth data; recognizing tiny objects; recognizing facial features;recognizing objects in a power constrained or network constrainedenvironment; and so on.

In embodiments, the dynamic vision system 11300 may integrate into orwith a set of value chain network (VCN) entities (such termsencompassing the many examples and embodiments disclosed herein and inthe documents incorporated by reference herein).

In embodiments, the dynamic vision system 11300 may be integrated intoor with a set of robotic systems, such as mobile and/or autonomousrobotic systems. For example, the dynamic vision system 11300 may becontained within the housing or body of a robotic system, such as amulti-purpose/general purpose robotic system, such as one that simulateshuman or other animal species capabilities. The vision capabilities mayenable the robot in identifying and manipulating a target object for usein robotic assembly lines where object depth, orientation, position andmotion may be inferred for improved object identification. The visioncapabilities may also enable the robot in simultaneous localization andmapping, which is a technique for estimating the position of the robotwith respect to its surroundings while mapping the environment at thesame time. As another example, the dynamic vision system 11300 may beintegrated with a robotic exoskeleton designed to augment thecapabilities of a human operator and provide optimized sensing andcontrol for the human operator.

In embodiments, the output from the dynamic vision system 11300 may betemporally combined with output from other sensors in the robot usingconditional probabilities to create a combined view of the object thatis richer and includes information about the position, orientation andmotion of the object. Some examples of sensors that may be used inconjunction with the liquid lens based dynamic vision system 11300include cameras, LIDARs, RADARs, SONARs, thermal imaging sensor,hyperspectral imaging sensor, illuminance sensors, force sensors, torquesensors, velocity sensors, acceleration sensors, position sensors,proximity sensors, gyro sensors, sound sensors, motion sensors, locationsensors, load sensors, temperature sensors, touch sensors, depthsensors, ultrasonic range sensors, infrared sensors, chemical sensors,magnetic sensors, inertial sensors, gas sensors, humidity sensors,pressure sensors, viscosity sensors, flow sensors, object sensors,tactile sensors, or some other type of sensor.

In embodiments, the dynamic vision system 11300 incorporating aconformable liquid lens controlled by AI as necessary, and augmented bysensors may be adapted to build a neural prosthetics system.

In embodiments, the dynamic vision system 11300 incorporating aconformable liquid lens technology controlled by AI as necessary, may beadapted to build an exoskeleton system.

In embodiments, the dynamic vision system 11300 incorporating aconformable liquid lens controlled by AI as necessary, and augmented bysensors may be adapted to perform facial recognition for human facesobscured by face masks.

FIG. 125 depicts a flow diagram illustrating a method for objectrecognition by the liquid lens based dynamic vision system according tosome embodiments of the present disclosure.

Referring to FIG. 125 , at 11402, real time data streams representingobject concept are received from the liquid lens based optical assembly.The data streams may be received at the sensor and include richcontextual and visual information generated by constantly adjustingliquid lens in response to changes in optical parameters. The datastreams may be analyzed at edge devices or sent to data processing bylocal or remote intelligence. The use of cloud-connectable edge devices,such as within computing infrastructure that is proximal to the dynamicvision system 11300 and/or that is integrated with or into the dynamicvision system 11300, such as where the dynamic vision system 11300 hasonboard edge computational and/or connectivity resources, such as 5G (orother cellular), Wi-Fi, Bluetooth, fixed networking resources, or thelike, may offer opportunities to provide rapid, real-time or nearreal-time processing responsiveness. At 11404, the real-time datastreams are processed by the image processing system to determine anobject concept that includes contextual intelligence about the objectand its environment. At 11406, the optical parameters are adjusted bythe control system leading to a change in configuration of the liquidlens. The constantly adjusting liquid lens creates real time datastreams at the sensor and rich metadata for image processing as theimages are based on additional optical parameters than just focal lengthand aperture. At 11408, the object concept is sequentially revised andused as an input to train a machine learning model, which dynamicallylearns on a training set of outcomes, parameters and data collected fromthe liquid lens based optical assembly. At 11410, the object conceptincluding contextual intelligence about the object and its environmentis utilized by artificial intelligence to make classifications,predictions, and other decisions relating to the object includingdetermining the position, orientation and motion of the object.

FIG. 126 depicts a schematic illustrating an example implementation of adynamic vision system for modelling, simulating and optimizing variousoptical, mechanical, design and lighting parameters of the dynamicvision system according to some embodiments of the present disclosure.The dynamic vision system may learn on data captured by sensors inresponse to sequentially adjusting the liquid lens to train theartificial learning system to use digital twins for classification,predictions and decision-making.

The digital twin system 11320 may be configured to simulate operation ofthe dynamic vision system 11300 so as to continuously capture the keyoperational metrics and may be used to monitor and optimize theperformance of the dynamic vision system 11300 in real-time, or nearreal-time. The digital twin system 11320 may create a digital replica ordigital twins 502 of one or more of the components or subsystems of thedynamic vision system 11300. The digital twins 502 of the one or more ofthe components or subsystems may use substantially real-time sensor datato provide for substantially real-time virtual representation and forsimulation of one or more possible future states of the one or morecomponents and subsystems. The digital twins 502 may be updatedcontinuously based on sensor data, to reflect the current condition orparameter values of the component or subsystem. The digital twins thusprovide a high fidelity, digital simulation of the behavior of thecomponent or subsystem. This capability may be used to produce a digitalprofile of both the prior and current behaviors of the component orsubsystem with the resulting profile used to detect behavior that isless than optimal as well as to predict future behavior of the componentor subsystem.

Referring to FIG. 126 , the digital twins 11502 in the dynamic visionsystem 11300 may include object twin 11504, environment twin 11506,liquid lens twin 11508, optical lens twin 11510, sensor twin 11512,process twin 11514, actuator twin 11516, object concept twin 11518 andthe like, that allow for modeling, simulation, prediction,decision-making, and classification by the processing system 11306. Thedigital twins 11502 may be populated with relevant data, for example theliquid lens twins 11508 may be populated with data related tocorresponding a liquid lens including dimension data, material data,shape data, feature data, thermal data, vibration data, and the like.The digital twins may provide one or more simulations of both physicalelements and characteristics of the one or more components or subsystemsbeing replicated and the dynamics thereof, in embodiments throughout thelifecycle of the one or more components being replicated.

In embodiments, the digital twins 11502 may provide a hypotheticalsimulation of the one or more components or subsystems, for exampleduring a design phase before the one or more components are manufacturedor fabricated, or during or after construction or fabrication of the oneor more components by allowing for hypothetical extrapolation of sensordata to simulate a state of the one or more components, such as duringany suitable hypothetical situation. In embodiments, the machinelearning model 11520 may automatically predict hypothetical situationsfor simulation with the digital twins 11502, such as by predictingpossible improvements to the one or more components, predicting if oneor more components are compatible with one another, predicting when oneor more components may fail and/or suggesting possible improvements tothe one or more components, such as changes to parameters, arrangements,configurations, or any other suitable change to the components. Forexample, the liquid lens twins 11506 and optical lens twins 11510 mayrun hypothetical simulations to check for compatibility with one anotheras well as with the optical assembly and predict the optimal arrangementin the assembly.

In embodiments, the machine learning models 11520 in conjunction withdigital twins 11502 may help drive various applications includingmaterial selection 11522, design optimization 11524, and motionprediction 11526.

In embodiments, the digital twins 11502 may allow for simulation of theone or more components during both design and operation phases of theone or more components, as well as simulation of hypothetical operationconditions and configurations of the one or more components byfacilitating observation, measurement and analysis of various metricsand then passing the insights onto the design or operational processesfor improvement of these processes.

The simulation system 11322 may set up, provision, configure, andotherwise manage interactions and simulations between and among digitaltwins 11502. Thus, the simulation system may help simulate, evaluate andoptimize the behavior and characteristics of various components andsubsystems of the dynamic vision system 11300 using the digital twins11502 of such components and subsystems.

In embodiments, the artificial intelligent system 11326 may beconfigured to execute simulations in the simulation system 11322 usingthe liquid lens twins 11508 and/or other digital twins 11502 availableto the digital twin system 214. For example, the processing system 11306may adjust one or more optical parameters of the liquid lens twin 11508.In embodiments, the artificial intelligent system 11326 may, for eachset of parameters, execute a simulation based on the set of parametersand may collect the simulation outcome data resulting from thesimulation. For example, the artificial intelligent system 11326 mayexecute simulations by varying the optical parameters of the liquid lenstwin 11506 to generate simulation outcomes in the form of object concepttwin 11518. During the simulation, the processing system 11306 may varythe focal length, fluid materials, specularity, color, environment, lensshape and any other parameters of the liquid lens twin 11506. Theoutcome data from such simulations in the form of object concept twins11518 in addition to other sensor data as well as data from othersources may then be used to train the machine learning models 11520 bythe machine learning system 11324.

In embodiments, the machine learning models 11520 may process the datareceived from sensors, including the event data and the state data todefine simulation data for use by the digital twin system 11320. Themachine learning models 11520 may, for example, receive state data andevent data related to a particular component of the dynamic visionsystem 11300 and perform a series of operations on the state data andthe event data to format the state data and the event data into a formatsuitable for use by the digital twin system 11320. For example, machinelearning models 11520 may collect data from one or more sensorspositioned on, near, in, and/or around the liquid lens to process thesensor data into simulation data and output the simulation data to thedigital twin system 11320. The digital twin system 11320 may then usethe simulation data to create the liquid lens twin 11506, the simulationincluding for example metrics including shape, material, focal length,specularity, environment, lighting, color, temperature, pressure, wearand vibration. The simulation may be a substantially real-timesimulation, allowing for a user of the dynamic vision system 11300 toview the simulation of the liquid lens, metrics related thereto, andmetrics related to parts thereof, in substantially real time. Thesimulation may be a predictive or hypothetical situation, allowing for auser of the dynamic vision system 11300 to view a predictive orhypothetical simulation of the liquid lens, metrics related thereto, andmetrics related to components thereof.

In embodiments, the machine learning models 11520 and the digital twinsystem 11320 may process sensor data and create a digital twin for a setof components to facilitate real-time simulation, predictive simulation,and/or hypothetical simulation of a related group of components.

The machine learning models 11520 may be algorithms and/or statisticalmodels that performs specific tasks without using explicit instructions,relying instead on patterns and inference. The machine learning models11520 may build one or more mathematical models based on training datato make predictions and/or decisions without being explicitly programmedto perform the specific tasks. In example implementations, machinelearning models may perform classification, regression, clustering,anomaly detection, recommendation generation, digital twin creationand/or other tasks.

In embodiments, the machine learning models 11520 may perform varioustypes of classification based on the input data. Classification is apredictive modeling problem where a class label is predicted for a givenexample of input data. For example, the machine learning models 11520can perform binary classification, multi-class or multi-labelclassification. In embodiments, the machine-learning model may output“confidence scores” that are indicative of a respective confidenceassociated with classification of the input into the respective class.In embodiments, the confidence scores can be compared to one or morethresholds to render a discrete categorical prediction. In embodiments,a certain number of classes (e.g., one) with the relatively largestconfidence scores can be selected to render a discrete categoricalprediction.

In embodiments, the machine learning models 11520 may output aprobabilistic classification. For example, the machine learning models11520 may predict, given a sample input, a probability distribution overa set of classes. Thus, rather than outputting only the most likelyclass to which the sample input should belong, the machine learningmodels 11520 can output, for each class, a probability that the sampleinput belongs to such class. In embodiments, the probabilitydistribution over all possible classes can sum to one. In embodiments, aSoftmax function, or other type of function or layer can be used to turna set of real values respectively associated with the possible classesto a set of real values in the range (0, 1) that sum to one. Inembodiments, the probabilities provided by the probability distributioncan be compared to one or more thresholds to render a discretecategorical prediction. In embodiments, only a certain number of classes(e.g., one) with the relatively largest predicted probability can beselected to render a discrete categorical prediction.

In embodiments, the machine learning models 11520 may perform regressionmodeling and related processes to provide output data in the form of acontinuous numeric value. As examples, the machine learning models 11520may perform linear regression, polynomial regression, logisticregression, nonlinear regression, or some other modeling process. Asdescribed, in embodiments, a Softmax function or other function or layercan be used to squash a set of real values respectively associated witha two or more possible classes to a set of real values in the range(0, 1) that sum to one. For example, the machine learning models 11520can perform linear regression, polynomial regression, or nonlinearregression. As examples, the machine learning models 11520 can performsimple regression or multiple regression. As described above, in someimplementations, a Softmax function or other function or layer can beused to squash a set of real values respectively associated with a twoor more possible classes to a set of real values in the range (0, 1)that sum to one.

In embodiments, the machine learning models 11520 may perform varioustypes of clustering. For example, the machine learning models 11520 mayidentify one or more previously-defined clusters to which the input datamost likely corresponds. In some implementations in which the machinelearning models 11520 performs clustering, the machine learning models11520 can be trained using unsupervised learning techniques.

In embodiments, the machine learning models 11520 may perform anomalydetection or outlier detection. For example, the machine learning models11520 can identify input data that does not conform to an expectedpattern or other characteristic (e.g., as previously observed fromprevious input data). As examples, the anomaly detection can be used forfraud detection or system failure detection.

In some implementations, the machine learning models 11520 may provideoutput data in the form of one or more recommendations. For example, themachine learning models 11520 may be included in a recommendation systemor engine. As an example, given input data that describes previousoutcomes for certain entities (e.g., a score, ranking, or ratingindicative of an amount of success or enjoyment), the machine learningmodels 11520 may output a suggestion or recommendation of one or moreadditional entities that, based on the previous outcomes, are expectedto have a desired outcome.

As described above, the machine learning models 11520 may be or mayinclude one or more of various different types of machine-learnedmodels. Examples of such different types of machine-learned models areprovided below for illustration. One or more of the example modelsdescribed below can be used (e.g., combined) to provide the output datain response to the input data. Additional models beyond the examplemodels provided herein can be used as well.

In some implementations, the machine learning models 11520 may be or mayinclude one or more classifier models such as, for example, linearclassification models; quadratic classification models; and the like.The machine learning models 11520 may be or may include one or moreregression models such as, for example, simple linear regression models;multiple linear regression models; logistic regression models; stepwiseregression models; multivariate adaptive regression splines; locallyestimated scatterplot smoothing models; and the like.

In some examples, the machine learning models 11520 may be or mayinclude one or more decision tree-based models such as, for example,classification and/or regression trees; chi-squared automaticinteraction detection decision trees; decision stumps; conditionaldecision trees; and the like.

The machine learning models 11520 may be or may include one or morekernel machines. In some implementations, the machine learning models11520 may be or may include one or more support vector machines. Themachine learning models 11520 may be or may include one or moreinstance-based learning models such as, for example, learning vectorquantization models; self-organizing map models; locally weightedlearning models; and the like. In some implementations, the machinelearning models 11520 may be or may include one or more nearest neighbormodels such as, for example, k-nearest neighbor classifications models;k-nearest neighbors regression models; and the like. The machinelearning models 11520 may be or may include one or more Bayesian modelssuch as, for example, naïve Bayes models; Gaussian naïve Bayes models;multinomial naïve Bayes models; averaged one-dependence estimators;Bayesian networks; Bayesian belief networks; hidden Markov models; andthe like.

In some implementations, the machine learning models 11520 may be or mayinclude one or more artificial neural networks (also referred to simplyas neural networks). A neural network may include a group of connectednodes, which also can be referred to as neurons or perceptrons. A neuralnetwork may be organized into one or more layers. Neural networks thatinclude multiple layers may be referred to as “deep” networks. A deepnetwork may include an input layer, an output layer, and one or morehidden layers positioned between the input layer and the output layer.The nodes of the neural network may be connected or non-fully connected.

The machine learning models 11520 may be or may include one or more feedforward neural networks. In feed forward networks, the connectionsbetween nodes do not form a cycle. For example, each connection canconnect a node from an earlier layer to a node from a later layer.

In some instances, the machine learning models 11520 may be or mayinclude one or more recurrent neural networks. In some instances, atleast some of the nodes of a recurrent neural network can form a cycle.Recurrent neural networks can be especially useful for processing inputdata that is sequential in nature. In particular, in some instances, arecurrent neural network may pass or retain information from a previousportion of the input data sequence to a subsequent portion of the inputdata sequence through the use of recurrent or directed cyclical nodeconnections.

In some examples, sequential input data may include time-series data(e.g., sensor data versus time or imagery captured at different times).For example, a recurrent neural network may analyze sensor data versustime to detect or predict a swipe direction, to perform handwritingrecognition, etc. Sequential input data may include words in a sentence(e.g., for natural language processing, speech detection or processing,and the like); notes in a musical composition; sequential actions takenby a user (e.g., to detect or predict sequential application usage);sequential object states; and the like.

Example recurrent neural networks include long short-term (LS™)recurrent neural networks; gated recurrent units; bi-direction recurrentneural networks; continuous time recurrent neural networks; neuralhistory compressors; echo state networks; Elman networks; Jordannetworks; recursive neural networks; Hopfield networks; fully recurrentnetworks; sequence-to-sequence configurations; and the like.

In some examples, the machine learning models 11520 may be or mayinclude one or more non-recurrent sequence-to-sequence models based onself-attention, such as Transformer networks.

In some implementations, the machine learning models 11520 may be or mayinclude one or more convolutional neural networks. In some instances, aconvolutional neural network may include one or more convolutionallayers that perform convolutions over input data using learned filters.

Filters may also be referred to as kernels. Convolutional neuralnetworks may be especially useful for vision problems such as when theinput data includes imagery such as still images or video. However,convolutional neural networks may also be applied for natural languageprocessing.

In some examples, the machine learning models 11520 may be or mayinclude one or more generative networks such as, for example, generativeadversarial networks. Generative networks may be used to generate newdata such as new images or other content.

The machine learning models 11520 may be or may include an autoencoder.In some instances, the aim of an autoencoder may learn a representation(e.g., a lower-dimensional encoding) for a set of data, typically forthe purpose of dimensionality reduction. For example, in some instances,an autoencoder may seek to encode the input data and the provide outputdata that reconstructs the input data from the encoding. Recently, theautoencoder concept has become more widely used for learning generativemodels of data. In some instances, the autoencoder may includeadditional losses beyond reconstructing the input data.

The machine learning models 11520 may be or may include one or moreother forms of artificial neural networks such as, for example, deepBoltzmann machines; deep belief networks; stacked autoencoders; and thelike. Any of the neural networks described herein may be combined (e.g.,stacked) to form more complex networks.

The machine learning models 11520 may include one or more clusteringmodels such as, for example, k-means clustering models; k-mediansclustering models; expectation maximization models; hierarchicalclustering models; and the like.

In some implementations, the machine learning models 11520 may performone or more dimensionality reduction techniques such as, for example,principal component analysis; kernel principal component analysis;graph-based kernel principal component analysis; principal componentregression; partial least squares regression; Sammon mapping;multidimensional scaling; projection pursuit; linear discriminantanalysis; mixture discriminant analysis; quadratic discriminantanalysis; generalized discriminant analysis; flexible discriminantanalysis; autoencoding; and the like.

In some implementations, the machine learning models 11520 may performor be subjected to one or more reinforcement learning techniques such asMarkov decision processes; dynamic programming; Q functions orQ-learning; value function approaches; deep Q-networks; differentiableneural computers; asynchronous advantage actor-critics; deterministicpolicy gradient; and the like.

In embodiments, the data processing system is implemented using a neuralnetwork to provide real-time, adaptive control of the dynamic visionsystem 11300 including object classification and determination of objectposition, orientation and motion.

In some embodiments, a neural network model may be used directly todetermine adjustments to optical parameters using training or learningof a neural network model. Initially, the model may be allowed to chooserandomly from a range of values for each input optical control parameteror action. If the sequence of optical control parameter adjustments oractions leads to an incorrect prediction/classification, it may bescored as leading to an undesirable (or negative) outcome. Repetition ofthe process using different sets of randomly chosen values for eachoptical control parameter or action leads to reinforcement of thosesequences that least to desirable (or positive) outcomes. Ultimately,the neural network model “learns” what adjustments to make to a set orsequence of optical control parameters or actions in order to achievethe target outcome i.e., a correct prediction or classification.

In embodiments, methods and systems described herein may use aconvolutional neural network (referred to in some cases as a CNN, aConvNet, a shift invariant neural network, or a space invariant neuralnetwork), wherein the units are connected in a pattern similar to thevisual cortex of the human brain.

The initial layers of the CNN (e.g., convolution layers), may extractlow level features such as edges and/or gradients from the input objectconcept 720. Subsequent layers may extract or detect progressively morecomplex features and patterns such as presence of curvatures andtextures in image data and so on. The output of each layer may serve asan input of a succeeding layer in the CNN to learn hierarchical featurerepresentations from data in the input object concept 720. This allowsconvolutional neural networks to efficiently learn increasingly complexand abstract visual concepts.

In embodiments, capsule networks may be employed to use fewer labeledtraining examples to achieve similar classification performance of CNNs.

In embodiments, transformer-based, encoder-decoder architectures usingattention mechanisms may be used in conjunction with or in place ofconvolutional neural networks.

FIG. 127 depicts a schematic view illustrating an example implementationof a dynamic vision system depicting a detailed view of variouscomponents along with integration of the dynamic vision system with oneor more third party systems according to some embodiments of the presentdisclosure. The dynamic vision system 11900 may include a liquid lensoptical assembly 11304 configured to capture data from various datasources 11902 including vision sensors 11904, feedback sources 11906providing outcome data from the machine learning system, environmentcontrol 11908 generating data in response to a change in environmentfactors including temperature, pressure, humidity, vibrations etc.,lighting control 11910 generating data in response to a change in sourcelighting including color, color temperature, timing (PWM), amplitudeetc. and data library 11912.

The data storage and management system 11914 may maintain a record ofstate and event data for various components and subsystems of thedynamic vision system 11300 such that any of the services, applications,programs, or the like may access a common data source (which maycomprise a single logical data source that is distributed acrossdisparate physical and/or virtual storage locations). The data storageand management system 11914 may include a memory subsystem for storageof instructions and data and a file storage subsystem providingpersistent storage for program and data files. Further, the storage andmanagement system 11914 may include capabilities such as dataallocation, data caching, data pruning and data management and access toand control of intelligence and data resources.

The processing system 11306 may process the data captured by liquid lensoptical assembly 11304 and stored in data storage and management system11914 to optimize and adjust the optical parameters in real time throughthe machine learning system 11324 and the artificial intelligence system11326, the digital twin system 11320 and the control system 11314 asdescribed in detail in FIGS. 123, 124, 125 and 126 , or elsewhereherein.

In embodiments, a set of applications 11916 may enable the dynamicvision system 11300 to present meaning information to a user and enablethe user perform specific vision tasks. Some examples of applicationsprovided on the dynamic vision system 11300 include particle filter11918, 3D model generation 11920, Location or motion prediction 11922,Visual SLAM 11924, defect detection 11926 and adversarial neural networkdetection 11928.

In embodiments, the dynamic vision system 11300 may integrate with oneor more third party systems 11930 through connectivity facilitiesincluding interfaces, network connections, ports, applicationprogramming interfaces (APIs), brokers, services, connectors, wrappers,containers, wired or wireless communication links, human-accessibleinterfaces, software interfaces, micro-services, SaaS interfaces, PaaSinterfaces, IaaS interfaces, cloud capabilities, or the like. Theconnectivity facilities may facilitate the transfer of data between thedynamic vision system 11300 and the one or more third party systems11930.

In embodiments, the dynamic vision system 11300 may integrate into orwith a set of value chain network (VCN) entities for quality controlinspections and sorting objects in a production assembly line orlogistics chain wherein the liquid lens is configured to quickly adjustfocus to accommodate for, recognize and sort objects located at variousworking distances or objects of different heights.

In embodiments, the dynamic vision system 11300 may integrate into orwith a set of autonomous vehicle systems to scan the vehicle environmentand monitor the distance between the vehicle from other objects on theroad.

In embodiments, the dynamic vision system 11300 may integrate into orwith an interactive head-mounted device configured to display virtualcontent with an electrically adjustable liquid lens for providing acorrection for the displayed content by adjusting the electricallyadjustable liquid lens.

In embodiments, the dynamic vision system 11300 may integrate into orwith an unmanned automotive vehicle (UAV) navigation system to helpcontrol the position or course of the UAV in three dimensions.

Some non-limiting examples of third party systems 11930 that mayintegrate with dynamic vision system 11300 for incorporating visioncapability include IoT system 11932, value chain network (VCN) system11934, manufacturing execution system (MES) 11936, robot/cobot system11938, automotive system 11940, 3D printing system, ophthalmic system,surgical system, microscopy system, exoskeleton system, prostheticssystem, biometrics system, quality management system (QMS), compliancesystem, certification system, and the like.

In embodiments, the integration of the dynamic vision system 11300 withthe more third-party systems 11930 takes into account the specific needsand requirements of the third party systems 11930 and may customizecertain components and applications of the dynamic vision system 11300based on such requirements. For example, when integrating with a 3Dprinting system, defect detection may be provided whereas integrationwith a robotic cleaning system may benefit from the inclusion of virtualSLAM 11924.

FIGS. 128-142 relate to various embodiments of a fleet managementplatform that is configured to configure fleets of robot operating unitsto perform a wide array of jobs. In some embodiments, a fleet managementplatform may be used a value chain entity that is leveraged by one ormore organizations. The fleet management platform may be a standaloneservice or may be incorporated as part of a larger multi-serviceoffering. In embodiments, the fleet management platform receives a jobrequest (e.g., from a client device) and identifies a set of tasks to beperformed in completion of the requested job. In response to determiningthe set of tasks, the fleet management platform may determine a robotfleet configuration that includes a set of robot operating units and mayassign robot operating units to the set of tasks. As used herein, arobot operating unit may refer to an individual robot, a team of robots,or a fleet of robots that operate to complete a task or set of tasks. Anindividual robot may refer to a special-purpose robot, multi-purposerobot, exoskeleton robot, robotic process automation software, or othersoftware-based bot, as discussed further below. As will be discussed, insome embodiments, the fleet management platform may define aconfiguration of one or more multi-purpose robots to perform arespective task or sub-task and/or to operate in a certain type ofenvironment as part of the fleet configuration. As will be discussed, amulti-purpose robot may be configured with various modules that allowthe multi-purpose robot to perform certain tasks. For instance, amulti-purpose robot may be provisioned with specialized chips thatenable the robot to perform intelligence tasks, specialized sensors fora job or environment, liquid lenses for enabling certain machine-visionfunctionality, specialized appendages that are task specific (e.g.,clamps, grippers, drills, lifts, and/or the like), and/or other modulesthat configure the multi-purpose robot to perform a certain task or setof tasks.

In some embodiments, the fleet management platform may define a set ofworkflows, wherein a workflow may define an order by which certain tasksor sub-tasks are performed and the robot operating unit(s) that is/areassigned to the respective task or sub-task. In some embodiments, thefleet management platform may perform workflow simulations toiteratively redefine fleet configurations and/or workflows tosubstantially optimize the operation of the robot fleet. For example,the fleet configurations and/or workflows may be iteratively adjusted toreduce costs, improve logistical efficiencies, reduce the overall jobtime, or the like. Once the fleet configuration and workflows arefinalized, the fleet management platform may deploy the fleet. In someembodiments, the fleet management platform may facilitate the logisticsinvolved with delivering robot operating units and/or robot components,and/or supporting resources to the job site(s). Furthermore, in someembodiments, the fleet management platform may leverage additivemanufacturing capabilities, such as 3D printers or other capabilitiesdescribed herein or in the document incorporated by reference herein, infurtherance of resource provisioning/logistics, such that items that arecapable of being 3D-printed in an efficient manner may be printed ratherthan shipped. In embodiments, the fleet management platform may monitorthe robot fleet while performing a job, including the status of robotoperating units, the performance of jobs, and the like. In some of theseembodiments, the fleet management platform may automate maintenance ofrobots and/or resources to ensure an efficient use of an availableinventory and/or to reduce downtime at job locations.

In some embodiments, the fleet management platform may support fleetdigital twins that depict the status of the robot operating units and/orthe job performance based on data received from the robot operatingunits or other suitable data sources, such as edge devices,environmental sensor systems, platform resources (e.g., logisticsplatforms, enterprise resource management platforms, customerrelationship management platforms, and/or the like), and/or othersuitable data sources. The digital twins served by the fleet managementplatform may be adapted for various uses. For example, in someembodiments, a digital twin may be configured to provide a real-timestatus of a job being performed by a fleet of robots. In this way, auser may be able to drill down in different areas of a job site to viewthe progress with a job. In some example embodiments, a digital twin maybe configured to provide a status of a robot fleet, including individualrobots within the fleet. In these examples, a user may drill down ontoindividual robots in a team or fleet of robots to view the status of therobots. For example, the user may view the battery life of a robot, theavailability of other energy sources, the location of a robot, themobility options for the robot, the productivity of a robot, taskcompletion status of a robot, maintenance alerts of a robot, and/or thelike. In some example embodiments, the fleet management platform mayserve environmental digital twins that depict the environment of a robotfleet with real-time information, such as locations of object and otherrobots, sensor readings of the environment, and the like. In theseembodiments, a user may leverage an environmental digital twin toprovide remote control commands to a robot, a team of robots, or a fleetof robots. For instance, a robot or team of robots may encounter anunidentified object in performance of a task and may be unable to make adecision relating to the task performance. In some embodiments, thefleet management platform may obtain relevant data (e.g., LIDAR data,video feeds, environment maps, and the like) which may be depicted in anenvironment digital twin. The user may view the current scenario in theenvironmental digital twin and may provide instruction to the robotfleet how to proceed given the scenario presented in the environmentaldigital twin. The foregoing are non-limiting examples of digital twinsthat may be used in connection with a fleet management platform andother examples are discussed below.

FIG. 128 illustrates an example environment of a fleet managementplatform 12000 (also referred to as “platform 12000”) according to someembodiments of the present disclosure. In some embodiments, a fleetmanagement platform 12000 may be used a value chain entity that isleveraged by one or more organizations. The fleet management platform12000 may be a standalone service or may be incorporated as part of alarger multi-service offering. In embodiments, a robot fleet managementplatform 12000 includes a fleet operations system 12002 a dataprocessing system 12030, and an intelligence layer 12004 (e.g., aplatform level intelligence layer 12004). In embodiments, the fleetoperations system 12002 configures and manages robot operating unitsand/or jobs that are performed by robot operating units 12040. As willbe discussed, a robot operating unit 12040 may refer to individualrobots, individual robot task assemblies 12050, robot fleets 12060,and/or robot fleet support units 12080.

In embodiments, the fleet operations system 12002 includes, but is notlimited to, a communication management system 12010, a remote-controlsystem 12012, a resource provisioning system 12014, a logistics system12016, a job configuration system 12018, a fleet configuration system12020, a job execution system 12022, human interface system 12024, and amaintenance management system 12026. In embodiments, the communicationmanagement system 12010 is configured to facilitate fleet managementplatform communications, including with elements external to the fleetmanagement platform 12000. In embodiments, the remote-control system12012 is configured to manage and enable control of robot operatingunits and fleet resources remotely. In embodiments, the resourceprovisioning system 12014 is configured to handle allocation and accessto fleet resources (e.g., robot operating units). In embodiments, thelogistics system 12016 coordinates use and transportation of fleetresources and supplies to job sites and/or robot operating units. Inembodiments, the maintenance management system 12026 facilitatescoordinated, timely maintenance of fleet resources. In embodiments, thejob configuration system 12018 generates a job execution plan based on ajob request. In embodiments, a fleet configuration system 12020configures robot operating units (e.g., individual robots and/or robotfleets) to complete a job execution plan. In embodiments, the jobexecution system 12022 executes, monitors, and/or reports on jobs beingperformed by robot operating units (e.g., in accordance with a jobexecution plan) to ensuring efficient use of fleet resources whileexecuting the job plan and addressing job and fleet related reportingrequirements. In embodiments, the human interface system provides aninterface by which a human user may interface with a robot operatingunit.

As mentioned, a robot operating unit 12040 may refer to individualrobots, individual robot task assemblies 12050, robot fleets 12060,and/or robot fleet support units 12080. In embodiments, individualrobots may include, but are not limited to, multi-purpose robots 12042,special-purpose robots 12044, exoskeleton robots 12046, and the like.FIG. 129 illustrates a non-limiting example set of components of amulti-purpose robot 12100 (MPR) and a special purpose robot 12180.

In embodiments, SPRs 12180 and MPRs 12100 may include a baseline system12102, a robot control system 12150, and a robot security system 12170.In embodiments, the robot control system 12140 includes a dataprocessing system 12130 and an intelligence layer 12140. As will bediscussed, the data processing system may include data processingresources that may be centralized and/or distributed amongst a team orfleet of robots. Additionally or alternatively, the data processingresources may include general purpose chipsets, specialized chipsets,and/or configurable chipsets. As will be discussed, the intelligencelayer 12140 performs intelligence related tasks on behalf of the robotor a collection of robots (e.g., a task assembly or fleet). For example,the robot-level intelligence layer 12140 may perform such tasks asartificial intelligence, machine-learning, natural language processing,machine vision, analytics, and/or the like and may leverage complex datastructures (e.g., digital twins) and disparate data sources (e.g., fromIoT, edge and other network-enabled devices, from on-premises andcloud-deployed databases and other resources, and/or from APIs, eventstreams, logs, or other data sources, among many others) in performancethereof. Robot-level and fleet-level intelligence layers are discussedin greater detail below. In embodiments, the robot security system 12170performs security related functions on behalf of a robot or a collectionof robots (e.g., a task assembly or fleet). These security-relatedfunctions may include autonomous adaptive and non-adaptive securityfunctions as well as manual security functions.

In embodiments, a baseline system 12102 of an MPR 12100 or an SPR 12190may include an energy storage and power distribution system 12104,enclosure 12106, an electro-mechanical and/or electro-fluidic system12108, a transport system 12110, a vision and sensing system 12112,and/or a structural system 12114. As will be discussed further below,the configuration of a baseline system of an SPR 12190 depends on thetypes of tasks that the SPR 12190 is configured to perform. Forinstance, the baseline systems of autonomous drones greatly differ fromthe baselines systems of autonomous vehicles or factory floor robots.Similarly, the baseline systems of MPRs 12100 depend on the type ofenvironments that the MPR 12100 is intended to operate in. For example,MPRs 12100 that are configured to operate in deep water conditions mayhave different baseline systems than MPRs 12100 that are configured tooperate in arctic conditions or aerial robots.

An MPR 12100 differs from an SPR 12190 in that a MPR 12100 can beconfigured to perform a wider range of disparate tasks. In embodiments,an MPR 12100 may further include a module system 12120 that allows anMPR 12100 to be configured with various hardware and/or softwarecomponents. In this way, an MPR 12100 may be fitted with differentappendages, sensor sets, chipsets, motive adaptors, and/or the likedepending on the range of tasks that the MPR 12100 is configured to do.In embodiments, the module system 12120 may include control moduleinterfaces 12130 and physical module interfaces 12122. The controlmodules interfaces 12130 and physical modules interfaces 12122 may referto mechanical, electrical, and/or digital interfaces that receiveauxiliary components to configure an MPR 12100 to perform certain tasks.In embodiments, the control module interfaces 12130 receive (orotherwise “connect” to) auxiliary components that alter one or morefeatures that relate to control of the MPR 12100. These may includechipsets (e.g., AI chipsets, machine-learning chipsets, machine-visionchipsets, communications chipsets, or the like), sensor modules,communication modules, AI modules, security modules, computing modules,and/or the like. In embodiments, the physical module interfaces 12122receive (or otherwise connect to) auxiliary physical modules that alterthe physical actions that may be taken by MPR 12100 and/or the physicaloperation of the MPR 12100. Examples of physical modules may include,but are not limited to, end effectors, motive adaptors, 3D printers,power supplies, and/or the like. As will be discussed, an MPR 12100 maybe reconfigured to perform one or more tasks in completion of a job. Inthese embodiments, the fleet management platform 12000 may define a jobexecution plan and a supporting robot fleet, and may provision one ormore modules to an MPR 12100 in the supporting robot fleet, such thatthe MPR 12100 is reconfigured to perform one or more specified tasks inthe job execution plan.

Referring back to FIG. 128 , individual robot task assemblies 12050 mayrefer to a collection of one or more individual robots that are assignedto perform a specific task or a set of related tasks. The robots in arobot task assembly may include any combination of MPRs 12042, SPRs12044, exoskeleton robots 12046, and the like. In some embodiments, anindividual robot task assembly 12050 may include a local manager thatcontrols or otherwise provides instructions to robots in the taskassembly 12050. In these embodiments, the local manager may be adesignated supervisor robot or a human operator. In embodiments, asupervisor robot may refer to a robot that is designated to organize,instruct, monitor, reassign, and/or reconfigure (or requestreconfiguration of) the robots in a task assembly 12050. In embodiments,the robot supervisor may act as an edge device on behalf of the taskassembly 12050, such that the robot supervisor may be allocated specificprocessing and/or communication capabilities that allow the robotsupervisor to communicate with the fleet management platform 12000 orother suitable devices or systems and/or to perform data processingoperations on behalf of the task assembly 12050. In embodiments, a robotfleet is a collection of individual robots and/or task assemblies thatcollectively perform a set of projects in completion of a job. Inembodiments, a robot fleet may include individual SPRs, MPRs,exoskeletons, and the like. Furthermore, fleets may be arranged as afleet of task groups, regional fleets, and/or a fleet of fleets. Inembodiments, a robot fleet may be supported by robot fleet support. Inembodiments, examples of robot fleet support may include on premises,edge and IoT devices, local data storages (and corresponding datainterfaces), maintenance support, charging stations and devices,replacement parts, batteries, accessories, shipping containers, dockingstations, spare parts, and/or technicians.

FIG. 130 illustrates the data processing system 12030 and theintelligence layer 12004 of the fleet management platform 12000. Inembodiments, the data processing system 12030 includes a data handlingservice 12032 and a data processing service 12034. The data handlingservice 12032 is configured to store, retrieve, and otherwise manage thedata of the fleet management platform 12000. In embodiments, the datahandling service 12032 accesses a set of data stores 12036 and/orlibraries 12038, whereby the data handling service 12032 writes andreads data from the data stores 12036 and/or libraries 12038 on behalfof other components of the fleet management platform 12000 and/or therobot operating units 12040. In embodiments, the data processing service12034 performs data processing operations on behalf of other componentsof the fleet management platform 12000 and/or the robot operating units12040. For example, the data processing service 12034 may performdatabase operations (e.g., table joins, retrieves, etc.), data fusionoperations, and the like. In embodiments, the data processing system mayinclude distributed resources, centralized resources, and/or “on-chip”resources.

In embodiments, the platform-level intelligence layer 12004 performsintelligence services on behalf of the other components of the fleetmanagement platform 12000 and/or the robot operating units 12040. Aswill be discussed, in some the platform-level intelligence layer 12004may be configured as part of a broader intelligence system (FIG. 131 ),whereby decision making and other intelligence-based functions areperformed at the lowest level possible. In embodiments, theplatform-level intelligence layer 12004 includes an intelligence layercontroller 12007 and a set of artificial intelligence services 12005. Inembodiments, the artificial intelligence services 12007 may include adigital twin system that manages and/or serves a set of digital twins(e.g., robot digital twins, robot team digital twins, robot fleetdigital twins, logistics digital twins, environment digital twins, andthe like. In embodiments, the artificial intelligence service 12007 mayinclude, link to, or integrate with a machine-learning (ML) system, arules-based intelligence system, an expert system, an analytics system,a robotic process automation (RPA) system, a machine vision system, anatural language processing (NLP) system, a neural network system and/orother intelligence or data handling system as noted throughout thisdisclosure or the documents incorporated herein by reference. Inembodiments, the intelligence controller 12007 includes an analysismanagement module, governance libraries, and analysis modules.

Intelligence Layer

FIG. 131 illustrates an example intelligence layer 12200 according tosome embodiments of the present disclosure. In embodiments, theintelligence layer 12200 is adapted from the intelligence services 8800of FIG. 104 to provide a framework for providing intelligence servicesat respective levels of a robotics-as-a-service ecosystem (e.g.,platform level intelligence layer 12004, a robot-level intelligencelayer 12140, or a fleet level intelligence layer (not shown)). In theseembodiments, the intelligence layer 12200 framework may be at leastpartially replicated in individual robots and/or at the fleet-level,such that an individual robot may leverage its intelligence layer 12200to attempt to generate decisions, recommendations, reports,instructions, predictions, classifications, or the like, while fleetlevel decisions, recommendations, reports, instructions, predictions,classifications, or the like may be made by one or more robots in thefleet, and platform level decisions, recommendations, reports,instructions, predictions, classification, or the like may be made by aplatform-level intelligence layer 12004. In these embodiments, requestsfor intelligence may be pushed to a higher level. For example, if arobot is unsure if there is an occluded object in its path, the robotmay escalate the request to the fleet level where one or more additionalrobots may work in connection with the robot to determine whether thisis an occlusion in the requesting robot's path. In another example, anunforeseen change in the environment (e.g., change in weather or otherconditions) may cause a robot fleet-level intelligence layer to alter ajob execution plan. In this example, the fleet level intelligence layermay not have enough information or processing resources to safely alterthe job execution plan. In response, the fleet-level intelligence layermay escalate the decision to the platform 12000-level intelligence layer12004, such that the platform 12000-level intelligence layer 12004 maydetermine a recommended alteration to the job execution plan given thechange in the environment.

In embodiments, the intelligence layer 12200 receives requests from aset of intelligence layer clients 12260. Depending on where within therobot fleet framework (e.g., fleet management platform-level,fleet-level, or robot-level) the intelligence layer 12200 isimplemented, intelligence layer clients 12260 may be various componentsof the fleet management platform (e.g., the remote control system 12012,the logistics system 12016, the job configuration system 12018, thefleet configuration system 12020, the job execution system 12022, and/orthe like), a robot fleet (e.g., one or more MPRs and/or SPRs in a teamor fleet), or individual robots (e.g., the robot control system of therobot, various modules of an MPR, and/or the like). In embodiments, anintelligence layer client 12260 provides an intelligence request to theintelligence layer 12200, whereby the request is to perform a specificintelligence task (e.g., a decision, a recommendation, a report, aninstruction, a classification, a prediction, a training action, an NLPrequest, or the like). In response, the intelligence layer 12200executes the requested intelligence task.

It is noted that in some scenarios, artificial intelligence services ofthe AI system 12204 may be intelligence layer clients 12260. Forexample, a rules-based intelligence system may request an intelligencetask from an ML system or a neural network system, such as requesting aclassification of an object appearing in a video and/or a motion of theobject. In this example, the rules-based intelligence system may be anintelligence layer client 12260 that uses the classification todetermine whether to take a specified action. In another example, amachine vision system may request a digital twin of a specifiedenvironment from a digital twin system, such that the ML system mayrequest specific data from the digital twin as features to train amachine-learned model that is trained for a specific environment.

In embodiments, an intelligence task may require specific types of datato respond the request. For example, a machine vision task requires oneor more images (and potentially other data) to classify objectsappearing in an image or set of images, to determine features within theset of images (such as locations of items, presence of faces, symbols orinstructions, expressions, parameters of motion, changes in status, andmany others), and the like. In another example, an NLP task requiresaudio of speech and/or text data (and potentially other data) todetermine a meaning or other element of the speech and/or text. In yetanother example, an AI-based control task (e.g., a decision on movementof a robot) may require environment data (e.g., maps, coordinates ofknown obstacles, images, and/or the like) and/or a motion plan to make adecision as to how to control the motion of a robot. In a platform-levelexample, an analytics-based reporting task may require data from anumber of different databases to generate a report. Thus, inembodiments, tasks that can be performed by an intelligence layerinstance may require, or benefit from, specific intelligence layerinputs 12270. In some embodiments, an intelligence layer 12200 may beconfigured to receive and/or request specific data from the intelligencelayer inputs 12270 to perform a respective intelligence task.Additionally or alternatively, the requesting intelligence layer client12260 may provide the specific data in the request. For instance, theintelligence layer 12200 may expose one or more APIs to the intelligencelayer clients 12260, whereby a requesting client 12260 provides thespecific data in the request via the API. Examples of intelligence layerinputs may include, but are not limited to, sensors that provide sensordata (e.g., robot sensors, environment sensors, and/or the like), videostreams (e.g., robot-captured video streams, video camera streams,and/or the like), audio streams (e.g., robot-captured audio streams,audio streams captured from an external microphone, and/or the like),databases (e.g., platform 12000 databases, third-party databases, and/orthe like), human input, and/or other suitable data.

In embodiments, an intelligence layer 12200 may include an intelligencelayer controller 12202 and an artificial intelligence (AI) service12204. In embodiments, an artificial intelligence layer 12200 receivesan intelligence request from an intelligence layer client 12260 and anyrequired data to process the request from the intelligence layer client12260. In response to the request and the specific data, one or moreimplicated services of the artificial intelligence service 12204 performthe intelligence task and the artificial intelligence service 12204outputs an “intelligence response”. An intelligence response may referto an output of the artificial intelligence service 12204. Examples ofresponses may include a decision made by an artificial intelligenceservice (e.g., a control instruction, a proposed job execution plan, aproposed fleet configuration, a proposed robot configuration, and/or thelike), a prediction made by an artificial intelligence service (e.g., apredicted meaning of a text snippet, a predicted outcome associated witha proposed action, a predicted fault condition and/or the like), aclassification made by an artificial intelligence service (e.g., aclassification of an object in an image, a classification of a spokenutterance, a classified fault condition based on sensor data), and/orother suitable outputs of an artificial intelligence service.

In embodiments, the artificial intelligence service 12204 may include anML system 12212, a rules-based system 12228, an analytics system 12218,an RPA system 12216, a digital twin system 12220, a machine visionsystem 12222, an NLP system 12224, and/or a neural network system 12214.It is appreciated that the foregoing are non-limiting examples ofartificial intelligence services, and some of the systems may beincluded or leveraged by other systems of the artificial intelligenceservice. For example, the NLP system 12224, the machine vision system12222, and the RPA system 12228 may all leverage different neuralnetworks in performance of their respective functions.

In embodiments, the intelligence services 12204 includes and providesaccess to a ML system 12222 that may be integrated into or be accessedby the fleet management platform 12000 or any sufficiently configuredrobot operating unit (e.g., an MPR, SPR, a team, a fleet, and/or thelike). In embodiments, the ML system 12212 may provide machine-basedlearning capabilities, features, functions, and algorithms for use by anintelligence system client 12260 such as training ML models, leveragingML models, reinforcing ML models, performing various clusteringtechniques, feature extraction, and/or the like. In an example, amachine learning system 12026 may provide machine learning computing,data storage, and feedback infrastructure to a workflow simulationsystem of a job configuration system to facilitate optimizing workflowdevelopment. The machine learning system 12026 may also operatecooperatively with other fleet intelligence systems, such as therules-based system, the machine vision system 12222, the RPA system12216, and/or the like.

In embodiments, the artificial intelligence services 12204 may includeand/or provide access to a neural network system 12214. In embodiments,the neural network system 12214 is configured to train, deploy, and/orleverage neural networks on behalf of an intelligence layer client12260. In embodiments, the neural network system 12214 may be configuredto train any suitable type of neural network that may be used by thefleet management platform 12000, a robot, a robot team, and/or a robotfleet. Non-limiting examples of different types of neural networks mayinclude any of the neural network types described throughout thisdisclosure and the documents incorporated herein by reference, includingwithout limitation convolutional neural networks (CNN), deepconvolutional neural networks (DCN), feed forward neural networks(including deep feed forward neural networks), recurrent neural networks(RNN) (including without limitation gated RNNs), long/short term memory(LTSM) neural networks, and the like, as well as hybrids or combinationsof the above, such as deployed in series, in parallel, in acyclic (e.g.,directed graph-based) flows, and/or in more complex flows that mayinclude intermediate decision nodes, recursive loops, and the like,where a given type of neural network takes inputs from a data source orother neural network and provides outputs that are included within theinput sets of another neural network until a flow is completed and afinal output is provided. In embodiments, the neural network system12214 may be leveraged by other components of the fleet intelligencesystem, such as the machine vision system 12222, the NLP system 12224,the rules-based system 12228, the digital twin system 12226, and/orother artificial intelligence services. Examples applications of theneural network system 12214 are described throughout the disclosure.

In embodiments, the artificial intelligence services 12204 may provideaccess to and/or integrate a robotic process automation (RPA) system12216. The RPA system 12216 may facilitate, among other things, computerautomation of producing and validating workflows that involveremote-control of robot operating units, teams, fleet resources and thelike. In embodiments, the RPA system 12216 may monitor human interactionwith various systems to learn patterns and processes performed by humansin performance of respective tasks. This may include observation ofhuman actions that involve interactions with hardware elements, withsoftware interfaces, and with other elements. Observations may includefield observations as humans perform real tasks, as well as observationsof simulations or other activities in which a human performs an actionwith the explicit intent to provide a training data set or input for theRPA system, such as where a human tags or labels a training data setwith features that assist the RPA system in learning to recognize orclassify features or objects, among many other examples. In embodiments,the RPA system 12216 may learn to perform certain tasks based on thelearned patterns and processes, such that the tasks may be performed bythe RPA system 12216 in lieu or in support of a human decision maker.Examples of the RPA systems 12216 may encompass those in this disclosureand in the documents incorporated by reference herein and may involveautomation of any of the wide range of value chain network activities orentities described therein. In embodiments, the artificial intelligenceservices 12204 may include and/or provide access to an analytics system12218. In embodiments, an analytics system 12218 is configured toperform various analytical processes on data output from fleetfunctional components, such as the fleet configuration system 12020,robot operating units, and the like. In example embodiments, analyticsproduced by the analytics system 12218 may facilitate quantification offleet system and system module performance as compared to a set of goalsand/or metrics. The goals and/or metrics may be preconfigured,determined dynamically from historical fleet operations results, and thelike. An analytics system 12218 may be confirmed to perform variousanalytics-based processes on behalf of the platform 12000, robot fleets,teams, and/or individual robots. Examples of analytics processes thatcan be performed by an analytics system 12218 are discussed below and inthe document incorporated herein by reference. In some exampleimplementations, analytics processes may include tracking goals and/orspecific metrics that involve coordination of supply chain activitiesthat may involve robotic capabilities (such as picking items andpreparing it for delivery by an autonomous vehicle, among many others)and demand intelligence, such as involving forecasting demand for a setof relevant items by location and time (among many others).

In embodiments, a value chain network analytic system may process a setof supply chain robotic fleet data and a set of demand intelligencerobotic process automation data to produce a recommended action thatcoordinates supply and demand for a set of goods or other items. Inembodiments, a value chain network automation system is provided thatincludes a supply chain robotic fleet data set including attributes of aset of states and capabilities of a set of robotic systems in a supplychain for a set of goods; a demand intelligence robotic processautomation data set including attributes of a set of states of a set ofrobotic process automation systems that undertake automation of a set ofdemand forecasting tasks for the set of goods; and a coordination systemthat provides a set of robotic task instructions for the supply chainrobotic fleet based on processing the supply chain robotic fleet dataset and the demand intelligence robotic process automation data set tocoordinate supply and demand for the set of goods.

In embodiments, the artificial intelligence services 12204 may includeand/or provide access to a the digital twin system 12220. The digitaltwin system 12220 may encompass any of a wide range of features andcapabilities described herein and in the documents incorporated hereinby reference. In embodiments, the digital twin system 12220 may beconfigured to provide, among other things, execution environments forand different types of digital twins, such as twins of physicalenvironments, twins of robot operating units, logistics twins, and thelike. In example embodiments, the digital twin system 12220 may furtherbe constructed to generate digital twins for fleet resources, jobaspects and the like, such as robot operating units assigned to a team;robot operating units in a fleet and the like. For example, the digitaltwin system 12220 may generate digital twins of robot resources (e.g.,exchangeable end effectors, power supplies, communication capabilities,motive adaptors, and the like). Further the digital twin system 12220may be configured with interfaces, such as APIs and the like forreceiving information from external data sources, such as data receivedfrom a physical robot operating unit and/or an environment thereof. Forinstance, the digital twin system 12220 may receive real-time data fromsensor systems of a robot operating unit and/or sensor systems of thephysical environment in which the robot operating unit operates. Inembodiments, the digital twin system 12220 may receive digital twin datafrom other suitable data sources, such as third-party services (e.g.,weather services, traffic data services, logistics systems anddatabases, and the like. In embodiments, the digital twin system 12220may include digital twin data representing features, states, or the likeof value chain network entities, such as supply chain infrastructureentities, transportation or logistic entities, containers, goods, or thelike, as well as demand entities, such as customers, merchants, stores,points-of-sale, points-of-use, and the like. The digital twin system12220 may be integrated with or into, link to, or otherwise interactwith an interface (e.g., a control tower or dashboard), for coordinationof supply and demand, including coordination of automation within supplychain activities and demand management activities.

In embodiments, the digital twin system 12220 may provide access to andmanage a library of robot operating unit digital twin systems. Systems,such as the other artificial intelligence services 12240 may access thelibrary to perform functions, such as a simulation of actions of a robotoperating unit in a given environment performing a specified job inresponse to certain stimuli. In embodiments, the digital twin system12220 may include and provide access to as well as facilitate executionof robot twins (e.g., digital twin of individual robot operating units),task twins (e.g., digital representation of tasks as defined by, forexample the task definition system and/or pre-configured library ofrobot task building blocks, which may be optimized for certain jobconditions/requirements), team twins (e.g., digital embodiment ofdesignated teams of robot operating units that may include digital twinsof individual robot operating units and the tasks that they areperforming and/or pre-configured task-range-specific team twins),project twins (e.g., digital embodiment of a defined job execution plan,optionally including digital twins for robot operating units, teams,tasks, fleet resources and/or a set of preconfigured project-specificproject twins that can address a range of specific tasks), fleet twins(e.g., an aggregation of robot operating unit digital twins along withfleet operational and organizational models that take into considerationcross-job fleet functions, such as maintenance, robot operating unitretirement and replacement, backup robot operating units and the like),operator twins (e.g., a digital embodiment of a human operator, such asmay be determined through use of robotic process automation and thelike), logistics twins (e.g., digital modeling for shipment and cost ofrobots, personnel, and support equipment—job independent and as neededfor addressing a particular job request), environment twins (e.g.,modeling mobility constraints and required capabilities, edge andnetworking constraints and capabilities, and power constraints andcapabilities), and the like.

In embodiments, the artificial intelligence services 12204 may includeand/or provide access to a machine vision system 12222. In embodiments,a machine vision system 12222 is configured to process images (e.g.,captured by a camera) to detect and classify objects in the image. Inembodiments, the machine vision system 12222 receives one or more images(which may be frames of a video feed or single still shot images) andidentifies “blobs” in an image (e.g., using edge detection techniques orthe like). The machine vision system 12222 may then classify the blobs.In some embodiments, the machine vision system 12222 leverages one ormore machine-learned image classification models and/or neural networks(e.g., convolutional neural networks) to classify the blobs in theimage. In some embodiments, the machine vision system 12222 may performfeature extraction on the images and/or the respective blobs in theimage prior to classification. In some embodiments, the machine visionsystem 12222 may leverage classification made in a previous image toaffirm or update classification(s) from the previous image. For example,if an object that was detected in a previous frame was classified with alower confidence score (e.g., the object was partially occluded or outof focus), the machine vision system 12222 may affirm or update theclassification if the machine vision system 12222 is able to determine aclassification of the object with a higher degree of confidence. Inembodiments, the machine vision system 12222 is configured to detectocclusions, such as objects that may be occluded by another object. Inembodiments, the machine vision system 12222 receives additional inputto assist in image classification tasks, such as from a radar, a sonar,a digital twin of an environment (which may show locations of knownobjects), and/or the like. In embodiments, the machine vision system12222 may output object classifications to an intelligence serviceclient 12260, such as a control system of a robot, a robot supervisor,an edge device, and/or the like. In some embodiments, a machine-learningsystem 12222 (e.g., of a robot operating unit) may include or interfacewith a liquid lens. In these embodiments, the liquid lens may facilitateimproved machine vision (e.g., when focusing at multiple distances isnecessitated by the environment and job of a robot) and/or other machinevision tasks that are enabled by a liquid lens.

In embodiments, the artificial intelligence services 12204 may includeand/or provide access to a natural language processing (NLP) system12224. In embodiments, an NLP system 12224 performs natural languagetasks on behalf of an intelligence layer client 12260, such as a controlsystem. Examples of natural language processing techniques may include,but are not limited to, speech recognition, speech segmentation, speakerdiarization, text-to-speech, lemmatization, morphological segmentation,parts-of-speech tagging, stemming, syntactic analysis, lexical analysis,and the like. In embodiments, the NLP system 12224 may enable voicecommands that are received from a human. In embodiments, the NLP system12224 receives an audio stream (e.g., from a microphone) and may performvoice-to-text conversion on the audio stream to obtain a transcriptionof the audio stream. The NLP system 12224 may process text (e.g., atranscription of the audio stream) to determine a meaning of the textusing various NLP techniques (e.g., NLP models, neural networks, and/orthe like). In embodiments, the NLP system 12224 may determine an actionor command that was spoken in the audio stream based on the results ofthe NLP. In embodiments, the NLP system 12224 may output the results ofthe NLP to an intelligence service client 12260, such as a controlsystem of a robot, a robot supervisor, an edge device, and/or the like.

In embodiments, the artificial intelligence services 12204 may alsoinclude and/or provide access to a rules-based system 12228 that may beintegrated into or be accessed by the fleet management platform 12000 orany sufficiently configured robot operating unit (e.g., an MPR, SPR, ateam, a fleet, and/or the like). In some embodiments, a rules-basedsystem 12228 may be configured with programmatic logic that defines aset of rules and other conditions that trigger certain actions that maybe performed in connection with a robot fleet and/or job. Inembodiments, the rule-based system 12228 may be configured withprogrammatic logic that receives input and determine whether one or morerules are met based on the input. If a condition is met, the rules-basedsystem 12228 determine an action to perform, which may be output to arequesting intelligence layer client 12260. The data received by therules-based engine may be received from an intelligence data source12270 and/or may be requested from another intelligence service 12204,such as the machine vision system 12222, the neural network system12214, the ML system 12212, and/or the like. For example, the rule-basedsystem 12228 may receive classifications of objects in a field of viewof the robot from the machine vision system 12222 and/or sensor datafrom a lidar sensor of the robot and, in response, may determine whetherthe robot should continue in its path, change its course, or stop. Therules-based system 12228 may be configured to make other suitablerules-based decisions on behalf of a respective client 12260, examplesof which are discussed throughout the disclosure. In some embodiments,the rules-based engine may apply governance standards and/or analysismodules, which are described in greater detail below.

In embodiments, the artificial intelligence services 12204 interfaceswith an intelligence layer controller 12202 is configured to determine atype of request issued by an intelligence layer client 12260 and, inresponse, may determine a set of governance standards and/or analysesthat are to be applied by the artificial intelligence service 12204 whenresponding to the request. In embodiments, the intelligence layercontroller 12202 may include an analysis management module 12206, a setof analysis modules 12208, and a governance library 12210.

In embodiments, an intelligence layer controller 12202 is configured todetermine a type of request issued by an intelligence layer client 12260and, in response, may determine a set of governance standards and/oranalyses that are to be applied by the artificial intelligence service12204 when responding to the request. In embodiments, the intelligencesystem controller 12202 may include an analysis management module 12206,a set of analysis modules 12208, and a governance library 12210. Inembodiments, the analysis management module 12206 receives a request foran artificial intelligence service and determines the governancestandards and/or analyses implicated by the request. In embodiments, theanalysis management module 12206 may determine the governance standardsthat apply to the request based on the type of decision that wasrequested and/or whether certain analyses are to be performed withrespect to the requested decision. For example, a request for a controldecision that results in a robot moving to another location mayimplicate a certain set of governance standards that apply, such assafety standards, legal standards, quality standards, or the like,and/or may implicate one or more analyses regarding the controldecision, such as a risk analysis, a safety analysis, an engineeringanalysis, or the like.

In some embodiments, the analysis management module 12206 may determinethe governance standards that apply to a decision request based on oneor more conditions. Non-limiting examples of such conditions may includethe type of decision that is requested, a jurisdiction in which a robotfleet, a geolocation in which a robot fleet is deployed, an environmentin which a robot fleet and/or robot operating unit is operating, currentor predicted environment conditions of the environment and/or the like.In embodiments, the governance standards may be defined as a set ofstandards libraries stored in a governance library 12210. Inembodiments, standards libraries may define conditions, thresholds,rules, recommendations, or other suitable parameters by which a decisionmay be analyzed. Examples of standards libraries may include, legalstandards library, a regulatory standards library, a quality standardslibrary, an engineering standards library, a safety standards library, afinancial standards library, and/or other suitable types of standardslibraries. In embodiments, the governance library 12210 may include anindex that indexes certain standards defined in the respective standardslibrary based on different conditions. Examples of conditions may be ajurisdictions or geographic areas to which certain standards apply,environmental conditions to which certain standards apply, robot typesto which certain standards apply, materials or products to which certainstandards apply, and/or the like.

In some embodiments, the analysis management module 12206 may determinethe appropriate set of standards that must be applied with respect to aparticular decision and may provide the appropriate set of standards tothe artificial intelligence service 12204, such that the artificialintelligence service 12204 leverages the implicated governance standardswhen determining a decision. In these embodiments, the artificialintelligence service 12204 may be configured to apply the standards inthe decision-making process, such that a decision output by theartificial intelligence service 12204 is consistent with the implicatedgovernance standards. For example, in operating a robot fleet in aparticular jurisdiction or geographic area, certain legal or regulatorystandards may be implicated, such as restrictions on types of robots(e.g., no drones), preservation of certain species or ecosystem (e.g.,protected wetlands), or the like. In this example, a decision regardinga fleet configuration may exclude certain types of robots from the fleetconfiguration (e.g., no drones) and may ensure that none of the robotsin the fleet pose a threat to the ecosystem in which the robot fleet isto operate. In another example, a control system of a robot may requesta control decision from the intelligence layer of the robot. In thisexample, the presence of humans or other living beings in proximity to arobot operating unit may implicate a set of standards (e.g., safetystandards, legal standards, or the like). In this example, theintelligence layer 12200 may receive inputs such as a video feed, LIDARdata, and the like. The AI service 12204 may initially classify anobject in the analysis management module 12206 may receive input fromthe video feed that indicates a human is in the field of view of therobot. In response, the analysis management module 12206 may determinethat certain safety standards applies and may provide the implicatedgovernance standards from the safety standards library to the AI service12204, which may then attempt to determine a control decision given aset of intelligence system inputs (e.g., current location, destination,video inputs, LIDAR data, and/or the like) and the implicated safetystandards. If the AI service 12204 cannot make a decision given thesafety standards, the AI service 12204 may issue a default decision(which may be defined in the safety standards library), such as stoppingand/or relinquishing control to a human operator. It is appreciated thatthe standards libraries in the governance library may be defined by theplatform 12000 provider, customers, and/or third parties. The standardsmay be government standards, industry standards, customer standards, orother suitable sources. In embodiments, each set of standards mayinclude a set of conditions that implicate the respective set ofstandards, such that the conditions may be used to determine whichstandards to apply given a situation.

In some embodiments, the analysis management module 12206 may determineone or more analyses that are to be performed with respect to aparticular decision and may provide corresponding analysis modules 12208that perform those analyses to the artificial intelligence service12204, such that the artificial intelligence service 12204 leverages thecorresponding analysis modules 12208 to analyze a decision beforeoutputting the decision to the requesting client. In embodiments, theanalysis modules 12208 may include modules that are configured toperform specific analyses with respect to certain types of decisions,whereby the respective modules are executed by a processing system thathosts the instance of the intelligence layer 12200. Non-limitingexamples of analysis modules 12208 may include risk analysis module(s),security analysis module(s), decision tree analysis module(s), ethicsanalysis module(s), failure mode and effects (FMEA) analysis module(s),hazard analysis module(s), quality analysis module(s), safety analysismodule(s), regulatory analysis module(s), legal analysis module(s),and/or other suitable analysis modules.

In some embodiments, the analysis management module 12206 is configuredto determine which types of analyses to perform based on the type ofdecision that was requested by an intelligence system client 12260. Insome of these embodiments, the analysis management module 12206 mayinclude an index or other suitable mechanism that identifies a set ofanalysis modules 12208 based on a requested decision type. In theseembodiments, the analysis management module 12206 may receive thedecision type and may determine a set of analysis modules 12208 that areto be run executed based on the decision type. Additionally oralternatively, one or more governance standards may define when aparticular analysis is to be performed. For example, the engineeringstandards may define what scenarios necessitate a FMEA analysis. In thisexample, the engineering standards may have been implicated by a requestfor a particular type of decision (e.g., a fleet configuration request)and the engineering standards may define scenarios when an FMEA analysisis to be performed (e.g., when the fleet is to operate in a certain typeof environment, such as underwater, underground, in enclosures, or whenworking with hazardous materials). Continuing this example, therules-based system 12228 of the AI service 12204 may determine that therequest corresponds to one of the defined scenarios and then may invokean FMEA analysis module to perform the analysis with respect to therequested decision.

When an artificial intelligence service 12204 is performing anintelligence task that implicates an analysis, the artificialintelligence service 12204 may execute the corresponding analysismodule(s) to analyze a potential decision determined with respect torequested intelligence task. If none of the implicated analysis modules12208 flag the decision as having violated the respective analysis, theartificial intelligence service 12204 may output the proposed decisionto the intelligence client 12260. If the proposed decision is flagged byone or more analysis modules 12208, the artificial intelligence service12204 may determine an alternative decision and may execute theimplicated analysis module(s) until a decision is reached.

In embodiments, an analysis module 12208 may leverage one or morestandards that are defined in one or more standards libraries that arestored in a governance library 12210. In some embodiments, standardslibraries may define conditions, thresholds, rules, recommendations, orother suitable parameters by which a decision may be analyzed. Examplesof standards libraries may include, legal standards library, aregulatory standards library, a quality standards library, anengineering standards library, a safety standards library, a financialstandards library, and/or other suitable types of standards libraries.In embodiments, a respective standards library may include an index thatindexes certain parameter sets defined in the respective standardslibrary based on different conditions. Examples of conditions may be ajurisdictions or geographic areas to which certain standards apply,environmental conditions to which certain standards apply, robot typesto which certain standards apply, materials or products to which certainstandards apply, and/or the like. In these embodiments, the analysismanagement module 12206 may determine the appropriate set of standardsthat must be applied to a particular decision, whereby the correspondinganalysis module is parameterizes with the determined standards, suchthat the parameterized analysis module 12206 performs the respectiveanalysis using these standards. In these embodiments, the analysismodules 12208 may be configured to apply different standards to the sameanalysis based on one or more conditions surrounding the decision.

In an example, before outputting a proposed control decision thatinstructs a robot to move forward is provided to a robot controller ofthe robot, an intelligence service 12204 of the robot may analyze aproposed decision with respect to a set of standards and/or rulescorresponding to the control decision. In this example, the artificialintelligence service 12204 may execute a safety analysis module and/or arisk analysis module and may determine an alternative decision if theaction would violate a legal standard or a safety standard. In anotherexample, before a fleet configuration proposal is output to therequesting client, an intelligence service 12204 of the fleet managementplatform 100 may analyze the proposed fleet configuration to ensure thatthe proposed fleet configuration does not violate any jurisdictionallegal or regulatory standards (e.g., certain types of robots may beprohibited from operating in certain areas or environments, certaincommunication protocols may be prohibited in certain areas orenvironments) and/or does not potentially threaten the quality of jobperformance (e.g., the selected configuration may include robots that donot perform well in certain conditions) and/or the condition of therobots (e.g., operating certain types of robots in unsuitableconditions, such as freezing temperatures, high humidity areas, salt orfresh water, and/or the like). In response to analyzing the proposeddecision, the artificial intelligence service 12204 may selectivelyoutput the proposed condition based on the results of the executedanalyses. If a decision is allowed, the artificial intelligence service12204 may output the decision to the requesting intelligence layerclient 12260. If the proposed configuration is flagged by one or more ofthe analyses, the artificial intelligence service 12204 may determine analternative decision and execute the analyses with respect to thealternate proposed decision until a conforming decision is obtained.

It is noted here that in some embodiments, one or more analysis modules12208 may themselves be defined in a standard, and one or more relevantstandards used together may comprise a particular analysis. For example,the applicable safety standard may call for a risk analysis that can useor more allowable methods. In this example, an ISO standard for overallprocess and documentation, and an ASTM standard for a narrowly definedprocedure may be employed to complete the risk analysis required by thesafety governance standard.

As mentioned, the foregoing framework of an intelligence system 12200may be applied at various levels of the disclosed environment. Forexample, in some embodiments, a platform level intelligence system(e.g., intelligence layer 12200) may be configured with the entirecapabilities of the intelligence system 12200, and certainconfigurations of the intelligence system 12200 may be provisioned forrespective robot operating units depending on the jobs assigned to therobot operating units. Furthermore, in some embodiments, a robotoperating unit may be configured to escalate an intelligence system taskto a higher level (e.g., the fleet level, edge device, or the platform12000) when the robot operating unit cannot perform the taskautonomously. It is noted that in some embodiments, an intelligencelayer controller 12200 may direct intelligence tasks to a lower levelcomponent. For example, the intelligence layer controller 12202 of arobot fleet or the fleet management platform 12000 may direct anintelligence request to an intelligence layer 12200 of a particularrobot provided the robot has access to the intelligence data sources12270 necessitated by the intelligence request. Furthermore, in someimplementations, an intelligence layer 12200 may be configured to outputdefault actions when a decision cannot be reached by the intelligencelayer 12200 and/or a higher or lower level intelligence layer. In someof these implementations, the default decisions may be defined in a ruleand/or in a standards library.

Security System

FIG. 132 illustrates an example of a security system 12280 according tosome embodiments of the disclosure. In embodiments, the security system12280 illustrates a framework that may be implemented at various levelsof the disclosed systems. In these embodiments, instances of thesecurity system 12280 may be implemented at the platform 12000-level, atthe fleet- or team-level, or individual-level. For example, at theplatform 12000-level, the security system 12280 may providesecurity-related functionality on behalf of the platform 12000 and/orwith respect to any communications and/or other interactions with robotoperating units. In embodiments, the security system 12280 implementedat the fleet-level or team-level, whereby the security-system may beconfigured to provide security-related functionality on behalf of therobot team or fleet and/or with respect to communications and/or otherinteractions with robots in the team or fleet. In embodiments, thesecurity system 12280 implemented at the robot-level may be configuredto provide security-related functionality on behalf of the robot and/orwith respect to communications and/or other interactions with otherrobots, robot teams, and/or the platform 12000.

In embodiments, the security system 12280 may include an autonomousadaptive security module 12282, an autonomous non-adaptive securitymodule 12284, and/or a manual security module 12286. An autonomousadaptive security module 12282 may be configured to request intelligencetasks from an intelligence layer 12200, whereby an adaptive securitymodule 12282 leverages the artificial intelligence services 12204 of anintelligence layer 12200 to assess a security risk and determine anaction based on an output of the intelligence layer 12200. For example,the adaptive security module 12282 of a robot fleet may monitor one ormore conditions associated with the robot fleet by receiving data from aset of data sources, such as monitoring a work area for potentiallydangerous conditions based on a set of data sources (e.g., video feeds,sensor data from the robots and/or environment, input from individualrobots, and/or the like). In response to receiving the data, theadaptive security module 12282 may request an assessment (e.g., aclassification) of an environment from the intelligence system 12200regarding the security of the environment. In response, the intelligencesystem 12200 may provide one or more classifications that indicate anassessment of the environment. The adaptive security module 12282 maythen determine whether the assessment necessitates an action to betaken, and if so, what particular action to take. In some of theseembodiments, the adaptive security module 12282 may use a rules-basedapproach to determine whether the assessment necessitates and actionand, if so, what action to take. Additionally or alternatively, theadaptive security module 12282 may leverage a neural network that istrained to an action to recommend given a set of features (e.g.,classifications, sensor readings from one or more robots, locations ofrobots, objects detected in the environment and locations thereof,and/or any other relevant features). In these embodiments, the neuralnetwork system 12214 may receive the features from the adaptive securitymodule 12282 and/or a set of intelligence layer inputs 12270 and mayoutput a proposed action given the set of features. In some of theseembodiments, an intelligence controller 12202 of the intelligence system12200 may allow or override decisions made by the artificialintelligence services 12204. For instance, the analysis modules 12208may perform dynamic risk analyses 12292 and/or static risk analyses12294. Examples of dynamic risk analysis may include, but are notlimited to, real-time data driven analyses (e.g., current weatherpatterns, current political climates, current health crises, and/or thelike) and/or job-specific risk analyses (e.g., contractual risks,environmental risks, safety liabilities, monetary liabilities, and/orthe like). Examples of static risk analyses may include, but are notlimited to, operational risks (e.g., product design risks, manufacturingrisks, quality control risks, and/or the like) and/orregulatory/compliance risks.

In embodiments, the autonomous adaptive security module 12282 mayoperate in an isolated manner (e.g., without communication with externaldevices or systems) or in a connected manner (e.g., with communicationwith external devices or systems).

In embodiments, the security system 12280 may include an autonomousnon-adaptive security module 12284. In embodiments, the autonomousnon-adaptive security module 12284 is configured to make securityrelated decisions on behalf of a client autonomously (e.g., withouthuman intervention). In embodiments, a non-adaptive security module12284 performs logic-based security-related actions (e.g., riskmitigation actions) in response to detecting one or more specific setsof conditions. For example, a non-adaptive security module 12284 may beconfigured to, in response to detecting a specific set of conditions,trigger actions such actions as turning off the robot, stopping amovement of the robot, initiating charging, sounding an alarm, sending anotification to another device or system, self-destructing, or the like.In embodiments, the non-adaptive security module 12284 responds to risksthat are more easily diagnosable, such as overheating conditions, movingor being taken out of a geofenced area, detected internal leaks, lowpower conditions, low fluid levels, and/or the like.

In embodiments, the security system 12280 may include a manual securitymodule 12286. In embodiments, the manual security module 12286 isconfigured to allow a user to make decisions regarding security-relatedactions. In some of these embodiments, the manual security module 12286is configured to receive a notification of an assessed risk (e.g., fromthe adaptive security module 12282, the non-adaptive security module12284, from an intelligence client 12260, or the like). In theseembodiments, the human user may interface with the manual securitymodule 12286 via a human interface, which may be provided via a userdevice (e.g., mobile device, tablet, computing device, or the like).

Various security and risk-mitigation strategies are discussed throughoutthe disclosure.

FIG. 133 illustrates an example set of components of the fleetoperations system 12002 of a fleet management platform. In embodiments,the fleet operations system 12002 may utilize the features andcapabilities of the robot fleet management platform 12000 to facilitatesubstantially optimized utilization of fleet resources by anticipatingfleet resource needs and preparing those resources in advance ofanticipated use. In embodiments, resource need anticipation may includecoordinating maintenance activities with job scheduling to ensure thatpreventable interruptions due to lack of maintenance are prevented.Additionally or alternatively, resource need anticipation may be basedon alignment of detected fleet resource use with information thatsupports, among other things, anticipation of job requests. Inembodiments, factors such as weather pattern forecasting, time of year,location, and/or the like may influence the likelihood of certain jobrequests (e.g., during hurricane season, urgent infrastructure repairjobs are likely to be requested). Example implementations for generatingfleet need predictions and addressing those predictions follow thediscussion of the components of the fleet operations system 12002 andthose of the related robot fleet management platform 12000. Aspreviously discussed, example components of the fleet operations system12002 may include a communication management system 12010, theremote-control system 12012, a resource provisioning system 12014, alogistics system 12016, a job configuration system 12018, a fleetconfiguration system 12020, a job execution, monitoring, and reportingsystem 12022 (also referred to as a “job execution system” 12022), and ahuman interface system 12024.

In embodiments, the communication management system 12010 is configuredto enable communication (e.g., efficient and/or high speedcommunication) among fleet management platform elements, such as thefleet operations system 12002 and its elements as described herein, thefleet intelligence layer 12004 and its elements as described herein,external data sources 12036, third party systems (e.g., via an Internetand the like), robot operating units, support systems and equipment,human fleet resources and the like. The communication management system12010 may include or provide access to one or more communication networktypes, such as wired, wireless and the like that may support variousdata protocols, such as Internet Protocol (IP) and the like. Thecommunication management system may include or have access tointelligence services (e.g., via the fleet intelligence system resourcesdescribed herein) that manage and control portions of the fleetmanagement platform infrastructure associated with communication toensure, for example: timely delivery of data collected by deployed robotoperating units to critical computation, analysis and/or data storageresources; prioritized delivery of robot configuration and operationalinstructions; and the like. In fleet resource management and controlembodiments, the communication management system 12010 may prioritizefleet security system communications use of fleet communicationresources over communications among fleet intelligence system componentsto support a high degree of security and integrity of fleet resources.The communication management system 12010 may provide and manage accessto networking, including fleet platform network system 380 that connectsat least the fleet management platform 12000 with external systems,deployed robot operating units, and other network-connectable elements(e.g., fleet edge devices and the like).

In embodiments, capabilities of the communication management system12010 may include contextual specification, and/or adaptation of robotfleet communication resources (e.g., networks, radio systems, datacommunication devices, such as routers, and the like) based on, amongother things, a job execution plan, plan definitions, task definitions,robot operating unit configurations, real-time job status, and the like.Communication management system 12010 adaptation of fleet communicationresources may be impacted by a range of real-world conditions (e.g.,weather, atmospheric conditions, building structures, workingenvironment (e.g., land-to-submerged, subterranean), and the like). Inembodiments, the communication management system 12010 may glean contextfrom a job request that may facilitate anticipating a need for and typesof adaptation during job execution. As an example of job requestcontext-based communication adaptation, a job may initiate at sea level,and then include actions by subterranean teams and high-altitude teams.Communication resources suitable for use in these different taskenvironments that are configured by the fleet configuration systemduring job configuration activities may be adaptively controlled by thecommunication management system 12010 for the respective teams of robotsas a job progresses through the exemplary environments.

Job request criteria may directly call for isolated operation.Alternatively, circumstances of the job request may favor isolatedoperation (e.g., operation within a foreign jurisdiction and the like).Communication resources for the requested job may be adaptedaccordingly. As an example, communication among a team of fleetresources assigned to co-locate when performing a job may be configuredby the fleet configuration system with additional encryption or with aradio frequency that defies conventional detection that thecommunication management system may facilitate activating when requiredby the job request (e.g., as noted above when the team enters a foreignjurisdiction). Further, communication outside of the team may be limitedby the communication system to certain locales, such as only when theentire team is located outside of a high-risk zone or other designation(e.g., within a building or the like). In this example, a courier robotmay be configured to travel from the co-location job site to a safeexternal communication site to exchange information with a remote fleetmanagement facility or the like and upon return to the co-location site,may use only communication processes and systems authorized for thatlocation. This non-limiting example describes a representative extent ofdiversity of communication capabilities and conditions to be handled bythe fleet communication management system. Isolated operation mayfurther or instead include no inter-robot operating unit communication,such as no wireless communication and the like as a condition of meetingjob request requirements and/or environmental limitations (e.g., workingin remote mountains or other isolated environments). In this furtherembodiment of fleet resource configuration, the communication managementsystem 12006 may detect and control communication resources (e.g., robotoperating unit radio interfaces, communication infrastructure that isproximal to isolated robot operating units and the like) to enforce sucha fleet configuration. Yet further consideration for isolated operationmay include adaptable isolation communication protocols, such aspermitting only use of low energy near-field communication conditionallybased on deployment context, such as an expected location of teamrobots, such as when multiple robot operating units are expected to benearby. The communication system 12006 may assist the fleetconfiguration system with fleet configuration, such as configuring robotoperating units, selection of robot units that meet a job requestcommunication requirement, configuration and designation of deploymentof fleet communication resources (e.g., co-locating an inter-robotoperating unit repeater device with the team), and other fleet and robotconfiguration considerations. In an example of such fleet configurationassistance, a job request may indicate a preference to use specificrobot operating units. The fleet configuration system may query thecommunication control system regarding adaptation capabilities (e.g., ofthe fleet communication management system and/or certain fleetcommunication resources) to support the preferred robot operating units.

In an example of communication management adaptability capabilities forsupporting diverse robot operating unit communication configurations,the communication management system 12010 may support a first team ofrobot operating units performing a field operation in using a differentradio frequency for wireless communication than a second team of robotoperating units that are performing field operations in the same radiosignal range as the first team of robot operating units; therebymitigating the likelihood of cross-radio interference. Further thecommunication management system 12010 may provide for reliablecommunication through use of redundancy, such as through dual radiosystems, automatic channel selection (e.g., local networking, cellularnetworking, mesh networking, long range satellite networking, and thelike). Fleet communication resources may include robot operating unitsacting as network elements, such as when robot operating units areconfigured into one or more mesh networks and the like. Robot operatingunits may facilitate communication in other ways, including visually,such as through use of light sources (e.g., Morse code or binarytransmissions), physical gestures, infrared signals, and the like.Auditory communications among robots (e.g., non-human language encodedaudio signaling), ultrasound and other auditory-based techniques may berendered as a form of communication among robots. Much like co-locatedrobots on different teams may use different radio frequency signals,co-located robots may use different auditory signaling to assist incommunication clarity among team members.

In embodiments, the communication management system 12010 may beconstructed as a plurality of independent communication systems that areconfigured to meet at least a corresponding portion of fleetcommunication needs. In an example, the communication management system12010 may be constructed with a first communication system forcommunicating among elements within the fleet operations system 12002(or any other fleet system, system, module, team, fleet segment and thelike), and with a second communication system for communication amongfleet intelligence layer elements (or any other portion of the fleetplatform that can be separated from the first communication system), sothat disruption of any individual communication system may be isolatedfrom other platform communication systems, thereby reducing impact ofcommunication problems throughout the platform 12000. Further in thisexample, the fleet operations system 12002 and its constituent elements(e.g., job configuration system 12018, and the like) may continue tocommunicate through the first communication system and indeed performall pertinent fleet operation functions (including communication withremotely deployed fleet robot operating units and the like) even thoughaccess to fleet intelligence layer elements, such as a machine learningsystem may be compromised due to problems with the second communicationsystem serving the fleet intelligence layer. Further the communicationmanagement system 12010 may include security features that effectisolation and shunning of platform systems, systems, system elements,communication systems and other platform resources that appear to becompromised due to malware or the like. Other independent communicationsystems include robot-to-robot communication systems, human-to-robotcommunication systems, emergency response communication systems, and thelike. Yet further independent communication systems may be based onaspects, such as confidentiality of information (e.g., negotiationsbetween a fleet management provider and a job requestor), fleetoperations oversight and the like. In embodiments, the communicationmanagement system 12010 may be constructed to provide role-based (or thelike) access to different communication systems. As an example, a jobexecution system executing a first requested job may not be providedaccess to certain resources based on geofence conditions (e.g., when theresource is outside of a designated region). In another example, a fleetoperations executive may be granted concurrent access to robot operatingunits allocated to different jobs for performing fleet supervisoryfunctions.

In addition to and/or instead of separated communication systems, thefleet communication management system 12010 may provide for redundancy(multi-frequency radios, and the like) to address exception conditionsthat may cause network compromise, may require overriding operationalcommunication channels for emergency use and the like.

In embodiments, the fleet communication management system 12010 mayprovide fleet resource-specific (e.g., individual robot operating unit)secure communication so that two fleet resources (e.g., two robotoperating units, a robot operating unit and a fleet monitoring system,and the like) may communicate securely. The fleet communicationmanagement system 12010 may further provide broadcast capabilities tosupport notification, update, alert, and other services. Broadcastcapabilities may be fleet-wide (e.g., a notice to all fleet resources toobserve daylight savings time), team-specific (e.g., an update to allteam members regarding role changes of team members), job-specific(e.g., an alert to fleet resources assigned to a job, which may includea plurality of robot teams, that the job is put on hold), fleet resourcetype-specific to address issues that concern certain types of fleetresources (e.g., such as fleet robot operating units, multi-purposerobot operating units, one or more types of special-purpose robotoperating units, robot operating units configured in supervisory roles),fleet support units, location-specific units (e.g., all units within aflash flood zone), and the like.

In embodiments, the fleet communication management system 12010 may useor manage job-specific communications elements together with other fleetmanagement platform features or services including, without limitation,the fleet security system 12006, the fleet network system 380, andvarious resources including Artificial Intelligence (AI) chipsets, dataencoders, communication spectrum frequencies, and the like. The fleetcommunication management system 12010 may work together with the fleetsecurity system 12006, such as by providing secure high-up-time accessto fleet and associated communication resources. As an example, a fleetsecurity system 12006 may utilize a portion of configured communicationchannels (e.g., wired inter-computer links, wireless networks, and thelike) that may be reserved by the communication management system forsecurity use. The portion may include physically dedicated elements(e.g., wired connections, wireless access points that operate over adedicated set of frequencies, and the like). In embodiments, providingdedicated wireless access may include prioritization of security systemaccess to existing wireless networks, such as by routing security systemdata packets, streams, and the like ahead of non-security systempackets. As another example, a communication management system mayallocate communication devices with greater battery energy (highercharge) and/or fixed power supply for security system use whileallocating lower power, lower energy, and/or rechargeable devices fornon-security system use. Security system communication resourcemanagement and control may be fleet-wide, job-specific, team-specific,deployment locale-based, geolocation-based, and the like. As an example,the fleet configuration system may specify a configuration of fleetcommunication resources for meeting a security aspect of a requestedjob. This configuration may be applied to fleet resources and maintainedby the communication management system for the duration of resourceparticipation in the requested job.

A further cooperative operation of fleet security system 12006 with thefleet communication management system 12010 may include managing accessby fleet resources to external resources (e.g., websites, and the like)as well as access by external resources to fleet resources. The fleetsecurity system 12006 may deploy security agents and the like to fleetresources based on allocation/configuration of those resources. As anexample, a firewall-type security function of the fleet security system12006 may be deployed at, among other things, access points managed bythe fleet communication management system to connect distinctjob-specific communication systems. The fleet communication system 12010may also support management of one or more fleet resources, such asmobile robot operating units, that are configured by the fleetconfiguration system to have access to multiple isolated communicationsystems (e.g., a hub type deployment that facilitates access amongisolated communication systems). The fleet security system 12006 mayenforce inter communication system access rights by deploying andoperating a centrally managed threat detection and management systemagent on such hubs.

In embodiments, the fleet communication management system 12010 may takeadvantages of intelligence capabilities of fleet resources, such asresources with artificial intelligence capabilities (optionally providedby AI-specific chips and chip sets and the like), to establish dynamiccommunication management functions that enrich and work with fleetsecurity capabilities to further reduce the likelihood of a successfulintrusion into a fleet communication system. As an example, AI-basedfunctionality deployed throughout at least portions of a fleet resources(e.g., individual robot operating units and the like) may be relied uponto detect local environments with increased risk of intrusion or otherthreat (e.g., based on contextual and historical informationrepresentative of such environments and the like) so that thecommunication management system, optionally in cooperation with thefleet security system 12006 may adapt fleet communication resources forreducing such risk.

The fleet communication management system 12010 may make use of and/orfacilitate control of use by others of the fleet network system 380. Asan example of management of the fleet network system 380, the fleetcommunication management system 12010 may treat the fleet network system380 as a resource to be managed for use by fleet resources forcommunicating, such as by determining and/or controlling which resourcesutilize the network, how resources using the network at the same timemay be coordinated, network loading limits for such resources, and thelike.

In embodiments, the fleet operations system 12002 includes aremote-control system 12012 that is configured to assist the jobexecution system 12022 and provide a framework for remotely controllingrobot operating units and other external resources to complete tasksand/or jobs. In embodiments, the remote-control system 12012 may managedefinition and use of control signals for remote operation of robotoperating units (e.g., multi-purpose, special purpose, exoskeleton,humans, and the like), fleet support units, external resources and thelike. Robot remote-control as enabled by the remote-control system 12012may include definition and management of local robot operating unit torobot operating unit control signaling, such as when a team supervisorrobot is directing one or more robot team members to perform tasks.Other examples of remote-control signal management may includehuman-to-exoskeleton signaling, robot-to-robot fleet support signaling,intra-team robot operating unit signaling, and the like.

In embodiments, the fleet operations system 12002 includes aremote-control system 12012 that is constructed to assist the jobexecution system 12022 and provide a framework for remotely controllingrobot operating units and other external resources to complete tasksand/or jobs. The remote-control system may manage definition and use ofcontrol signals for remote operation of robot operating units (e.g.,multi-purpose, special purpose, exoskeleton, humans, and the like),fleet support units, external resources and the like. Robotremote-control as enabled by the remote-control system 12012 may includedefinition and management of local robot operating unit to robotoperating unit control signaling, such as when a team supervisor robotis directing one or more robot team members to perform tasks. Otherexamples of remote-control signal management may includehuman-to-exoskeleton signaling, robot-to-robot fleet support signaling,intra-team robot operating unit signaling, and the like. In embodiments,the remote-control system uses resources of the fleet managementplatform 12000 and/or a fleet configuration, including, for example, thefleet communication management system 12010, the fleet security system12006, and/or fleet network system 380 to access information, in somecases make decisions, and execute commands. The framework for remotelycontrolling robot operating units may comprise a series of actions basedstandard rules, adapted rules modified by situational awareness,emergency rules, exceptions, human decisions, ethical rules, the fleetintelligence system, etc. However, specialized, fall-over, or othercommunications necessary to handle a range of remote-controlrequirements may be part of the communication management system 12010that may facilitate delivery of remote control communication/signalingwhile what the communications should be versus may be determined fromuse of the remote-control system 12012.

The remote-control system 12012 may recognize a plurality of initiatorsof remote-control signals, including local supervisor remote-controlinitiators, human (local or remote) remote-control initiators, automatedfleet-based remote-control initiators (e.g., fleet artificialintelligence system and the like), third-party remote-control initiators(e.g., for law enforcement and the like). Remote control signaling mayinclude managing remote control signals to fleet-external resources,such as fire and emergency response resources, infrastructure resources,third-party robot service providers, and the like.

The fleet resources that may participate in remote-control operationsmay be diverse in both implementation and protocols, such as oldergeneration robot operating units, human fleet resources, quantumcomputing elements and the like. Therefore, the remote-control system12012 (in cooperation with the communication system 12006) may beconstructed with knowledge of multiple remote operational protocol(multi-protocol) capabilities to ensure any two devices exchangingcontrol signals can do so reliably. In embodiments, multi-protocolcapabilities may include handling and/or providing as a serviceprotocol-to-protocol translation, remote-control signal consolidationand interpretation, protocol normalization, and the like. Inembodiments, the communication management system 12010 may utilize theseprotocol handling capabilities directly as noted above and by API andthe like, or by being configured with such protocol handlingcapabilities (e.g., deployed with protocol handling capabilities of theremote-control system 12012. In embodiments, the remote-control system12012 (or equivalent functions thereof integrated with the communicationmanagement system 12010) may rely on portions of the fleet intelligencelayer, such as digital twin and/or artificial intelligence service, tofacilitate, for example protocol translation and/or adaptation.Therefore, the remote-control system 12012 may provide real-time, ondemand protocol translation, optionally assisted by the fleetintelligence system. The remote-control system 12012 may supportfleet-external remote-control via a port that is configured forintegration with external and/or third-party remote-controlarchitectures. Remote-control may be communicated via dedicatedinfrastructure and/or communication features (e.g., short-distancebroadcast capabilities).

A remote-control system 12012 may include an ethics capability that mayprovide guidance and/or regulation of remote control based on ethicsfactors, such as ensuring that a robot does no harm to humans, animals,the environment, and the like. Ethics factors may be influenced bygovernment and/or industry regulations, human behavior models thatfacilitate determining fairness, and the like. Ethics may be enforcedthrough statistical measures, such as based on voting by member of ateam of robot operating units. As an example of statistics-based ethicsenforcement, an action to override a job execution plan, an attempt at aremote takeover of a robot operating unit, or any other exception may beevaluated by a portion of team members wherein each member of theportion may contribute a perspective on the remote operation. Eachperspective may be a vote for/against permitting/taking theremote-control action. A robot operating unit vote may be split amongpossible outcomes (e.g., 90% for, 10% against) and the like to enact aform of weighting of the perspective on possible outcomes. Theremote-control system 12012 may be constructed to be influenced byethics-based decision making, such as robot operating unit voting asdescribed herein. Ethics-based control, and the like, may be combinedwith other remote-control system 12012 control capabilities so thatfactors beyond ethics, such as cost, and the like may be factored intoremote-control. In embodiments, ethics capabilities may be leveraged viathe intelligence layer 12200. In these embodiments, remote controlinstructions may be analyzed using one or more analysis modules 12208and/or and with respect to one or more set of governance standards.

Remote-control, such as control of robot operating units may beinitiated, at least in part, by a human operator. In embodiments, afleet operations system 12002 may encounter unexpected and/or unknownconditions during job execution (e.g., as may exemplarily be reported bythe job execution system 12022) and defer to a human operator toremotely control robot operating unit(s). Optionally one or more fleetintelligence system 12022 components, such as an artificial intelligencesystem may be referenced for at least candidate remote-control signals.In embodiments, a job execution plan may indicate, at a predeterminedoperational task, that robot operation should be guided by a humanoperator. When such a task is anticipated to occur in a job workflow(e.g., by a job execution monitoring instance, such as a supervisorrobot and the like), the remote-control system 12012 may be called uponto oversee a remote-control connection between a suitable human operatorand the robot, robot operating units, team, team supervisor and the likeexecuting the workflow that calls for human operator control.

In embodiments, the remote-control system 12012 may have access to a setof remote-control signal sequences for performing certain tasksremotely. The system 12012 may, based on context of a workflow beingperformed, suggest to a human operator and/or an automated controlsystem one or more remote-control signal sequences. In embodiments, theremote-control system may process input from a human operator (e.g.,commands such as “stop”, “evacuate” and the like), optionally with helpof other fleet resources (e.g., an artificial intelligence system andthe like) and generate a set of remote-control signals for remotelycontrolling fleet resource, such as a robot operating unit and the like.Remote control signal sequences may be preconfigured for handling arange of real-time situations, such as security breaches, equipmentfailure, and the like. In addition to facilitating and/or managingremote-control of a robot operating unit, remote-control signalsequences may be used for reconfiguration of fleet resources deployedand/or allocated for a task, workflow, job and the like. In an exampleof use of remote-control signals for reconfiguration, a set of robotoperating units performing a task may be remotely controlled to take ona new role due to failure of one of the robots in the set. A humanoperator (or an automated system monitor-type application) may provideremote control signals that are communicated to the viable members ofthe team to adjust task roles and actions accordingly, such as bycommunicating a remote control signal to one or more of the viablemembers to communicate with a robot operating unit configuration serverto receive reconfiguration instructions and reconfiguration data.

Although generally described herein as remote-control signals, theremote-control system 12012 may facilitate remote-control by arrangingremote control signals into remote control instructions (e.g.,combinations of remote-control signals, abstractions thereof and thelike) at the fleet level, team level, robot level and the like. As anexample of remote-control instruction functionality, the remote-controlsystem 12012 may receive input, such as from a human operator desiringto instruct all robots with illumination capability to activate lightstoward a target location to assist with optical inspection or some othervisual function that would benefit from greater illumination. In thisexample, the remote-control system may receive the human operatorremote-control instruction, adapt that instruction into one or moredifferent remote control signals for the robot operating units 12040that are within an illumination proximity of the target location andgenerate corresponding remote-control signals for each of the types ofproximal robot operating units and ensure communication of those signals(e.g., via the communication management system 12010 resources) to therobot operating units to be remotely controlled by the human operator.Yet further, robot operating units that receive the remote-controlinstruction may further participate in the implementation of theinstructions by, for example, communicating among the set receiving thesignals (and/or a subset thereof) to determine which, if any, robotoperating units are executing the instruction. A first robot thuslycontacted may be performing a time-sensitive function that would bedisturbed if it redirected its resources to providing the commandedillumination. By coordinating with other robot operating units, thefirst robot may continue with the time-sensitive function based onresponse(s) from other robot operating units regarding executing theremote-control command. In another remote-control instruction example, ateam of robot operating units may be remotely controlled by instructingthem, via the remote-control signals of the remote-control system 12012,to adjust operation for achieving reduced sound pollution (e.g.,activate a quiet mode of operation) for a period while a team of humaninspectors tour the job location where the team is operating. In anotherremote-control instruction example, a job-wide, team-wide, fleet-wide orother resource-specific remote control instruction may be issued toadjust an image presented on a display screen of the fleet resource(s)to reflect a changed logo due to acquisition of the fleet, temporaryassignment of the fleet resource(s), change in fleet messaging and thelike.

Robot operating unit responsiveness to aggregated remote-control signals(e.g., instructions or set of instructions) may be based on a wide rangeof fleet intelligence capabilities, knowledge, priorities, goals, andthe like. In general, use of platform-based and/or robot operatingunit-based artificial intelligence capabilities supports widerindependent decision-making capabilities for individual robot operatingunits with greater contextual gravity.

In embodiments, the remote-control system 12012 may integrate securityfeatures to thwart takeover, compromise, misuse or interference withcontrol of remotely controlled robot operating units. Resources used bythe remote-control system 12012 (e.g., data storage resources, computingresources, remote-control system state data and the like) may beconfigured with security features, such as encoding, decoding,packetizing, and the like. Further, the remote-control system 12012 mayinclude and/or support control override capabilities that enable a humanoperator (for example) to securely gain remote-control of a robot thatis otherwise not directly engaged with remote-control signaling or inother words operating independently of remote-control signals, such asautonomously, collaboratively with other robot operating units and thelike.

Resource Provisioning

In embodiments, the fleet operations system 12002 includes a resourceprovisioning system 12014 that manages provisioning resources for robotoperating units in a fleet, such as provisioning resources for robotteams, robot fleets, multi-purpose robots, and/or supporting resources(e.g., edge devices, communication devices, additive manufacturingsystems (e.g., 3D printers), and the like). In embodiments, resourcesmay include physical resources, digital resources, and/or consumableresources. Examples of physical resources may include, but are notlimited to, such as end effectors/manipulators, environmental shieldingcomponents, sensors and/or sensor systems, companion resources (e.g.,drones, transportation resources and the like), hardware resources(e.g., specialized processing modules, data storage, networking modules,tethering modules, and the like), spare parts, human resources (e.g.,technicians, operators, and the like), power sources (e.g., generators,portable batteries, and the like). Non-limiting examples of digitalresources may include software, operating parameters, job-specific datasets, and the like. Non-limiting example of consumable resources mayinclude fuel, sample collection containers, welding supplies,washdown/cleanup supplies, deployable resources (e.g., flares, safetycones, fall-zone netting and the like), and many others.

In embodiments, the resource provisioning system 12014 may provisionphysical resources from an inventory of physical resources, such asfleet-specific inventories, regional public-use inventories,rental/per-use fee-based resource inventories, on-demand resourceproduction systems (e.g., 3D printing of end effectors and the like),third party inventories, and the like. In some embodiments, the dataprocessing system 12030 maintains an inventory database in one or moredatastores 1203X. In embodiments, the inventory database storesinventory records, where each inventory record may indicate a respectiveresource (e.g., an identifier of the resource and/or of the type ofresource), the general availability of the resource (e.g., is itavailable, when is it available, etc.), pricing data relating to theresource, and other relevant data. For instance, for physical resourcessuch as robot units (e.g., SPRs, MPRs, and/or exoskeletons), hardwarecomponents, end effectors, and other physical components, an inventoryrecord may indicate an item identifier (e.g., a unique identifier thatidentifies the resource and/or a type of the resource), location of thephysical resources, a physical status of the physical resource (e.g., acondition of the physical resource, a maintenance record of the physicalresource, a predicted condition of the resource, etc.), ownership data(e.g., who owns the resource, is the resource buyable or leasable,etc.), a make and/or model of the physical resource, operational data(e.g., functions, intended conditions and environments, weight limits,speed limits, and the like), configuration data (e.g., systemrequirements, interface requirements, connectivity requirements), and/orthe like. In some embodiments, the inventory may include resources thatcan be 3D printed. In these embodiments, the inventory records mayadditionally or alternatively include printing requirements (e.g., 3Dprinters that can print the resource, materials needed to print theresource, etc.), printing instructions that define instructions for 3Dprinting, and/or other relevant information. In embodiments, theinventory records may provide inventories of digital resources, such assoftware products, middleware, device drivers, libraries, data feeds,microservices, and the like. In these embodiments, the inventory recordsmay indicate data relating to the digital resource, such as anidentifier of the digital resource, a provider of the digital resource,compatibility information relating to the digital resource, accessinformation (e.g., APIs, webhooks, and/or other information foraccessing or interfacing with the digital resource), pricinginformation, functionality of the digital resource, and/or the like. Aswill be discussed, the data processing system 12030 may be configured toreceive requests from the resource provisioning system 12014 (or othersuitable components, such as the fleet configuration system 12020) todetermine available inventories, inventory statuses, inventory pricing,and/or the like. In embodiments, the resource provisioning system 12014may query the data processing system 12030 to determine the availabilityof certain resources, the pricing of certain resources, the locations ofcertain resources, the statuses of certain resources, and/or the like.Additionally or alternatively, in some embodiments, the resourceprovisioning system 12014 (or another component, such as the fleetconfiguration system 12020) may query the data processing system 12030with a desired functionality of a resource, an intended use of a robotoperating unit (e.g., individual robot and/or fleet), an intendedenvironment of a robot, and/or compatibility requirements of a robotoperating unit. In response, the data processing system 12030 may returninventory records resources that correspond to the request.

In embodiments, the resource provisioning system 12014 may workcooperatively with other systems of the fleet operations platform, suchas fleet configuration systems, fleet resource scheduling andutilization systems, and the like to ensure fleet resource provisioningrules are followed. Physical resources to be provisioned may alsoinclude computing resources, such as on-robot computing resources, robotoperating unit-local fleet-controlled computing resources,cloud/third-party based computing resources, computing and other modulesand chips (e.g., for deployment with/within a robot operating unit), andthe like. In some embodiments, the fleet resource provisioning rules maybe defined in governance standards libraries, such that the resourceprovisioning system 12014 interfaces with the intelligence layer toensure that provisioned resources comply with the provisioning rules.

In embodiments, digital resources to be provisioned by the resourceprovisioning system 12014 may be provisioned through fleet configurationcapabilities, such as software/firmware update pushing (e.g., to updatea robot's on-board software), resource access credentialing (e.g., toaccess network resources, such as job-specific robot configuration dataand the like), on-robot data storageconfiguration/allocation/utilization data, and the like. In embodiments,consumable resources to be provisioned by the resource provisioningsystem 12014 may be sourced from a wide range of sources includingspecialized supply chains, job requestor resources (e.g., an office setup job may include use of job requestor-supplied office materials,worker personal materials and the like), job, team and/or fleet specificstockpiles. An example of job-related stockpiling includes stockpilingorange safety cones proximal to a long-term construction site that areaccessed by local robot operating units through the resourceprovisioning system 12014. Use of a provisioning system 12014 mayinclude provisioning equipment, material, software, data structures, andthe like (e.g., customized end effector) that are made and/or sourcedspecifically for a given job request.

In embodiments, the provisioning system 12014 may further operatecooperatively with contract systems, such as third-party smart contractsystems, and the like. In some embodiments, a job description mayreference or comprise a smart contract that may include and/or result inconfiguration of an instance of the provisioning system 12014 that iscompliant with the job description. As an example, a provisioning system12014 may receive, such as from a job configuration system 12018, smartcontract terms that call out provisioning constraints and/or guidance.The provisioning system 12014 may interpret these contract terms,thereby producing a set of fleet and consumable resource provisioningconstraints.

While the examples described above for a provisioning system 12014generally focus on job execution-related provisioning, the provisioningsystem 12014 may further handle provisioning of fleet resources, such ascomputing resources, access to and/or execution of fleet elements, suchas a fleet configuration system, intelligence layer, and the like. Inembodiments, provisioning of certain resources may be enacted as part ofa negotiation workflow for acceptance of a job request. As an example,provisioning certain intelligence services (e.g., a fleet levelintelligence layer) may result in a higher charge to a job requestorthan other intelligence services (e.g., only a robot-level intelligencelayer being deployed robot operating units). As noted above andelsewhere herein, intelligence services can bring value to the fleet andjob configuration functions of the platform 12000; thereforeprovisioning such systems as part of a job request negotiation mayjustify the additional cost to the job requestor.

In some scenarios, prioritization of the platform 12000 resources, suchas a fleet configuration system, may impact provisioning systemfunctions. If a job request only supports (e.g., based on price paid forthe job) use of such a fleet resource during off-peak hours, theplatform 12000 resource may not be provisioned to the job during peakhours, even if the platform 12000 resource is available.

In embodiments, the fleet operations system 12002 includes a logisticssystem 12015 that handles, among other things, logistics planning andexecution for meeting job requirements, maintaining robots, maintainingavailability of fleet resources (robot operating units, physicalresources, and the like), pickup and delivery of parts (e.g.,replacement parts, end effectors, supplies, and the like). In someembodiments, the logistics system 12015 may be configured to identifyavailability and locality of 3D printing resources to satisfy demandthat otherwise might not be feasible through conventional logistics(e.g., truck-based) transport means. In embodiments, the logisticssystem 12015 can leverage intelligence services, such as machinelearning systems and/or artificial intelligence systems to recommendlogistics plans.

Logistics plans may refer to a workflow that is generated to result inthe delivery of a set of items to a particular location. In embodiments,the logistics system 12015 may generate logistics plans that utilizefleet resources, such as transport type robots for execution of alogistics plan. Other than fleet resources may be utilized, such ascommon carriers, for-hire over-the-road truckers, private deliverycouriers, and the like. A determination of which resource to use forexecution of a logistics plan may be based on costs and availability ofresources. For example, the logistics system 12015 may determine thatthere are available fleet resources in a vicinity of a job that wouldnot require a third-party trucking service to deliver the availableresources from a remote location and, in response, the logistics system12015 may select the available resources over the third-party truckingsolution. In embodiments, the fleet operation system 12002 may leveragethe (platform-level) intelligence layer 12004 to assist in logisticsplanning and decision-making.

In embodiments, the fleet operations system 12002 includes a maintenancemanagement system 12026 that may be configured to schedule andeffectuate maintenance for fleet resources, such as robot operatingunits. A maintenance management system 12026 may handle fieldmaintenance needs and requests, including scheduled maintenance of fleetrecourses in the field to mitigate impact on robot operating unitutilization due to travel from a deployed job site to a repair depot.The maintenance management system 12026 may also coordinate maintenanceand repair operations at repair depots, and the like. Further themaintenance management system 12026 may work cooperatively with otherplatform systems, such as a logistics system 12015 to cause maintenanceto be performed during transport of a fleet resource, such as a robotoperating unit, between job sites. In embodiments, a maintenancemanagement system 12026 may include, provide access to, and/or beintegrated with mobile maintenance vehicles, spare parts depots,third-party maintenance service providers and the like. In embodiments,maintenance needs for fleet resources housed in storage areas, such aswarehouses, remote inventory depots and the like may be evaluated by themaintenance management system 12026 for pre-scheduled maintenance, suchas when a preventive maintenance activity for a robot is upcoming sothat the robot is less likely to require maintenance during adeployment.

In embodiments, the maintenance management system 12026 may monitor thestate of the fleet resources, such as robot operating units via resourcestate reports that may be provided on a scheduled basis or in responseto an inquiry for robot operating unit state by the maintenancemanagement system 12026 and the like. In embodiments, the maintenancemanagement system 12026 may monitor robot operating unit communicationfor an indication of a potential service condition, such as a robotoperating unit signaling to a supervisor robot that it is experiencingreduced power output, a robot operating unit reporting exposure tocertain ambient conditions (e.g., excessive heat), a lack of heartbeatsignal from a robot operating unit to a robot health monitor resource,and the like. Further, a maintenance management system 12026 may deployprobes within robot operating and/or supervisory software that mayperform maintenance management functions on a robot operating unit, suchas monitoring information in a robot data store that stores robotoperating unit state information, activating self-test operating modes,collection of data that provides indications of robot maintenance needsand the like. Yet further a maintenance management system 12026 mayinclude maintenance robots that may be deployed with other robots in ateam of robot operating units for performing a requested job. Amaintenance robot may be a configuration of a multi-purpose robotdeployed with a robot team. Such a configuration may be temporal withinbounds of a team deployment. A multi-purpose robot deployed forperforming tasks of a job workflow may be reconfigured dynamically (andoptionally temporarily) while deployed to a team to perform maintenanceactions on other robots and fleet resources.

A maintenance management system 12026 may be constructed to takeadvantage of a range of platform services and capabilities to scheduleand effectuate maintenance, including leveraging human/operator input(e.g., a human observer may indicate that a robot operating unit appearsto be operating erratically), robotic process automation of maintenanceactivities, artificial intelligence for predicting maintenance instancesfor scheduling, machine learning to help identify new opportunities forscheduling and performing maintenance (e.g., analyze performance ofrobot operating units that have been maintained for certain conditionsbefore performing certain tasks under those conditions, such asreplacing air filters before performing tasks in a dusty environment),and the like. In embodiments, a maintenance management system 12026 mayreceive maintenance related input. Maintenance related input may includemaintenance requests from robot operating units (for the requestingrobot operating unit or for another robot operating unit, such as acompanion robot operating unit). Maintenance related input may includerequest from or for maintenance of edge devices (e.g., fixedinfrastructure devices, fleet resources, job site proximal and/orjob-specific edge devices, such as edge devices deployed at a job siteby a job requestor and the like). Other candidate sources of maintenancerelated input may include supervisor robot operating units, humanoperators/observers, maintenance scheduling services, third-partyservice providers, robot production vendors, and parts providers toschedule maintenance. The maintenance management system 12026 may alsoleverage business rules (e.g., rules established for a team, fleet, by ajob requestor, determined by a regulatory agency and the like),association tables, data sets, databases, and/or maintenance managementlibraries to determine appropriate maintenance workflows, serviceactions, needed parts and the like. In embodiments, a maintenanceactivity may be assigned by the maintenance management system to a fleetresource, such as a maintenance robot, a human technician, a third-partyservice provider and the like.

In embodiments, robot operating units that are deployed may beconfigured with one or more maintenance protocols to perform, amongother things self-maintenance, such as calibrating end effectoroperations, adjusting tensioning structures to maintain a high degree ofmobility, and the like. Self-maintenance may include, withoutlimitation, reduction in capabilities responsive to detection of acompromised robot operating unit feature, such as a rotating mechanismthat no longer rotates continuously through 360 degrees. A deployedrobot operating unit may determine that a capability is compromised and,optionally with support of the maintenance management system 12026, mayswap assignments with another robot so that the compromised capabilitycan be resolved when time permits rather than causing a delay incompletion of a task. Also, robot operating unit intelligence (e.g.,on-robot AI and the like) may predict a compromise in robot capabilitiesbased on, for example, time-to-failure data for the robot capability. Ifthe time of this predicted compromise lands within a target taskperformance timeframe, the robot operating unit may call for pre-emptivemaintenance to be performed while the robot operating unit is in transitto a job site. The maintenance management system 12026 may process thiscall for maintenance and coordinate maintenance resources to beavailable during transit, and/or at a job site when the robot operatingunit is expected to arrive.

In embodiments, the maintenance management system 12026 may leverage theintelligence services of an intelligence layer 12200 (e.g., the platform12000 level intelligence layer 12004) to predict when maintenance may beperformed for robot operating units and/or components thereof. In someof these embodiments, the maintenance management system 12026 mayrequest a digital twin of a robot operating unit from the intelligencelayer 12200. In these embodiments, the digital twin may reflect acurrent condition of the robot operating unit, such that the robotoperating unit digital twin may be analyzed to determine whethermaintenance is required for the robot operating unit. Additionally oralternatively, the digital twin service of the intelligence layer 12200may run one or more simulations involving the robot operating unit topredict when maintenance may be required. In some of these embodiments,outputs of the digital twin of the robot operating unit may be analyzed(e.g., using a machine-learned prediction model or a neural network) topredict if/when maintenance may be required.

In embodiments, the fleet operations system 12002 includes a jobconfiguration system 12018. In embodiments, a job configuration systemreceives job requests, such as from customers that request a job. Inembodiments, a job request may indicate a set of job request parameters.Non-limiting examples of job request parameters may include: types ofprojects and tasks (e.g., inspection tasks, packaging tasks, unloadingtasks, loading tasks, shipping tasks, assembling tasks, monitoringtasks, digging tasks, construction tasks, delivery tasks, or the like),budget, timeline, environment description (e.g., indoors/outdoors, sizeof the environment, communication capabilities of the environment,layouts/blueprint/digital twin of the environment, or the like),location (e.g., region, address, coordinates, or the like), and anyother suitable parameters. In embodiments, the job request parameterswhich may be indicative of what types of robot operating units areneeded and/or functionalities thereof. These and other job requestdetails are described elsewhere herein.

In embodiments, the job configuration system 12018 may utilize a jobrequest to define a job configuration as a set of projects that are tobe completed in performance of a job, which may be ordered in ajob-level workflow. For each project, the job configuration system 12018may define a workflow that defines a set of tasks that are done incompletion of a project. In determining the job configuration, the jobconfiguration system 12018 may determine the projects, workflows, andtasks using a combination of techniques and resources including: (i)artificial intelligence techniques to define the projects, workflows,and/or tasks; (ii) libraries that can define default configurations ofdifferent types of jobs and/or projects; (iii) robotic processautomation; (iv) intelligence services (e.g., deep learning); and (v)quantum optimization.

In embodiments, quantum optimization may be enabled by a quantumoptimization system 12008 that may optimize task assignment across fleetresources, such as robot operating units and the like. A quantumoptimization system 12008 may further optimize routing (logical,physical, and electronic) associated with robot fleets, jobs, team,communications, logistics and the like. Additionally or alternatively,in some embodiments a quantum optimization system 12008 may be employedto optimize combinations of robotic resource with other resources acrossa variety of fleet functions including workforce diversity, energyconsumption, computational capacity and utilization, infrastructureresource planning, engagement and utilization, risk management,computing storage capacity, and the like.

In embodiments, a job configuration system 12018 and other fleetresources (e.g., fleet configuration, platform intelligence, robotoperation and the like) may benefit from use of deep learning techniquesfor task, workflow, and job execution plan optimization as well as forlearning, among other things, from failures. In these embodiments, thejob configuration system 12018 may request deep learning services fromthe platform 12000-level intelligence layer 12004, which leveragesneural networks and/or other machine-learned models to determine jobconfigurations based on a set of features, including features extractedfrom a job request. In these embodiments, the artificial intelligenceservices may be configured to learn task workflows, job configurations,and the like.

In embodiments, job configuration, fleet configuration (which mayinclude robot configuration), and/or as job execution may furtherenhance fleet functions, performance, and outcomes through use of localcontext-adaptive task assignment, execution, resource routing and thelike. This adaptive capability may be further enabled throughpeer-to-peer based communication (e.g., robot operating units within ateam) that reveals context of job activities rapidly and efficiently.

In embodiments, artificial intelligence for automation of multi-purposerobot task assignment and execution (e.g., robotic process automationthrough learning) may function cooperatively with elements of the fleetmanagement platform 12000, such as a fleet operations system 12002 andplatform intelligence layer 12004, to learn robot assignment from, forexample, human operator assignment activity. Other learning that anartificial intelligence system may yield in context of robot fleetconfiguration and operation may be based on outcome measures of successincluding task completion, time to completion, cost of completion,quality of completion, ROI for resources, resource utilization, andothers.

These and other job configuration details, including operational flowsof the job configuration system 12018 are depicted and described inrelated figures herein.

In embodiments, a fleet operations system 12002 includes a fleet androbot configuration system 12020 (also referred to as fleetconfiguration system 12020) that may work cooperatively with a jobconfiguration system 12018 to determine configurations of fleetresources (e.g., robot operating units, teams, and the like) to satisfyjob requests from a plurality of concurrent and/or overlapping jobrequests. The fleet configuration system 12020 may determine fleet androbot configurations based on job requests, projects, robot tasks, abudget, a timeline, availability of robots or robot types, theconfigurability options of multi-purpose robots, and/or other suitableconsiderations. As an example, fleet configuration may includespecifying a quantity of each type of robot that can be configured perjob, project, task or other unit of configuration. In some embodiments,the fleet configuration system 12020 may leverage the platform12000-level intelligence layer 12004 to determine fleet and/ormulti-purpose robot configurations. In some of these embodiments, theintelligence request may include a proposed job configuration and otherrelevant data (e.g., budgetary constraints, location, environment,etc.). In response, the intelligence layer 12004 may output a proposedfleet configuration (which may include multi-purpose robotconfigurations). Further details of a fleet configuration system 12020are described and depicted in figures elsewhere herein.

In embodiments, a fleet operations system 12002 may include a jobexecution, monitoring, and reporting system 12022 (also referred to as ajob execution system 12022). A job execution system 12022 may receive ajob execution plan from the job configuration system 12018 that itprocesses by coordinating activities of platform functions, such aslogistics for robot and fleet resource delivery, data processing system12030 allocation for facilitating data collection, cataloging, librarymanagement and data processing activities for job execution. In general,the job execution system 12022 may start a job with committing andmanaging resources, including resources beyond those configured by thejob configuration system 12018, such as computing, storage, bandwidth,and the like as may be defined by and/or determined to be useful forexecuting the job execution plan.

In embodiments, the job execution system 12022 may further facilitateadherence to reporting requirements (e.g., job-specific, fleet-specific,compliance-related reporting, and the like) associated with jobexecution. In embodiments, reporting may include data collection (e.g.,from robot operating units, sensor systems, user devices, databases,and/or the like), data processing, and feedback preparation for use ofjob execution data by job and fleet configuration systems and the like.In embodiments, the job execution system 12022 may be assisted by otherplatform capabilities that transmit, process, store, and manage datathat impacts job execution, such as the maintenance management system12026, the resource provisioning system 12014, and the communicationmanagement system 12010 that facilitates communications among robotoperating units, teams, and fleets, and others. These and other fleetand external resources may provide information to the job executionsystem 12022 for facilitating operational aspects of a requested job,such as which communication resources has the fleet communicationmanagement system 12010 reserved and/or allocated for the requested job,service and/or maintenance requirements for robot operating unit andother resources being used to execute a job, changes to resourceprovisioning that occur after operation of a job has commenced, and thelike.

In embodiments, the job execution system 12022 may further facilitateevaluation and modification of a job execution plan while executing thejob by, for example identifying bottle necks that are developing due toon-the-job conditions (e.g., traffic jams, ground conditions not asexpected due to excessive rain, and the like).

In embodiments, the job execution system 12022 may perform a variety ofdata pipeline functions during execution of a job. In embodiments, datapipeline functions may include, among other things, optimizing use ofpreconfigured sensor and detection packages that combine sensorselection, sensing, information collection, preprocessing, routing,consolidation, processing, and the like. In embodiments, sensor anddetection packages may be activated by the job execution system 12022when use thereof is indicated as serving a range of monitoring/reportingactivities. Other data pipeline function examples include optimizingon-robot storage, selective sensor data filtering for reduced impact oncommunication bandwidth (e.g., reducing the demand for wireless networkutilization), exception condition detection and pipeline adaptation/datafiltering, and others.

In embodiments, the job execution system 12022 may monitor, and ifnecessary, address robot power demand during job execution. In theseembodiments, the job execution system 12022 may ensure, for example,battery charge capacity (or other energy source levels, such as fuellevels) across multiple robot operating units to meet job task andworkflow requirements, such as a queue of tasks that should not beinterrupted. In embodiments, robot power demand management may includefleet, team, and individual robot operating unit routing to completetasks with reduced delays in overall productivity with integrated robotcharging activities. Further details of the functions and operation ofthe job execution system 12022 are described throughout the disclosure.

In embodiments, fleet functionality, including during job execution maybe combined with 3D printing services and systems to enable, forexample, agile, remote, flexible manufacturing on an as-demanded basisthrough, for example, deployment and use of optionally automated robotic3D printing and production capabilities proximal to a point of use(e.g., a job site, a logistics site, a warehouse, a transportationvehicle, and the like). Another exemplary use of fleet robotfunctionality with 3D printing combines this agile flexible productioncapability with customizable product delivery for last-milecustomization of products. Several exemplary embodiments of 3D printingfunctionality combined with the methods and systems of fleet managementare described elsewhere herein, including, without limitation on-robot3D printing of service items at a service site; 3D printing of jobspecific end-effectors and/or adaptors based on context acquired at ajob site; robot control of transportable (e.g., job site-deployed) 3Dprinting systems; 3D scanning and in-situ printing, and the like.

In embodiments, the job execution system 12022 may execute, deploy,and/or interface with a set of smart contracts that monitor and reporton robot operating units 12040. In embodiments, robust distributed datasystems, such as distributed ledgers (e.g., public or privateblockchains) can be utilized for tracking and enhancing robot fleetsand/or multi-purpose robot activities, as well as allocation of roboticresource utilization cost to relevant parties, such as job requestors,fleet users, and the like. In some of these embodiments, the distributedledger nodes store and execute smart contracts. In embodiments, thesmart contracts may be configured to monitor job requests, jobexecution, resource use, and/or the like. For example, in someembodiments, robot operating units may be configured to provide evidenceof completion of a task to a smart contract, such that the smartcontract may trigger actions (e.g., payments, recordation, or the like)in response to completed tasks. In another example, robot operatingunits may be configured to report location data, sensor data, statusdata (e.g., charge levels, component status, or the like), and/or othersuitable data, whereby the smart contract may be configured to triggercertain actions based on the received data.

In embodiments, a fleet operations system 12002 may include a dataprocessing system 12030 that may provide, among other things, access toscalable computation capabilities for any fleet operations and/orintelligence resources, data management capabilities (e.g., datacaching, storage allocation and management and the like), access to andcontrol of fleet and/or job-related data stores, such as libraries,fleet resource inventory control and management data structures and thelike.

In embodiments, the fleet operations system 12002 may include a humaninterface system 12024 that provides a human interface that allows usersto access the fleet management platform 100 and/or individual robotoperating units (e.g., for remote control) from a remote device (e.g., auser device, a VR device, an AR device, and/or the like). Inembodiments, the human interface system 12024 facilitates job requestentry (including any job-related parameters), fleet operationsmanagement, fleet resource management, fleet computing system, softwareand data structure management (e.g., system upgrades and the like),human access to robot operating units (e.g., for remote control of arobot operating unit), augmented and/or virtual reality visualizationsof fleet operation, data extraction (e.g., for generation of and/orvalidation of smart contracts associated one or more job requests andthe like). As an example of use of a human interface system 12024, a jobrequestor may access status updates of a requested job via the humaninterface system 12024. The job requestor may use a remote device toobserve robot operating units performing tasks for the requested job. Inthis example, the human interface system 12024 may interact with otherfleet components, such as the job execution system 12022, to directimage capture resources (e.g., camera-based overhead drones) to provideimages of robot operating units assigned to and currently performing jobtasks.

In embodiments, the fleet operations system 12002 may provide supportfor satisfying job requests. For example, the components of the fleetoperations system 12002 may facilitate resource provisioning andlogistics to ensure that fleet resources (e.g., robot operating units,physical modules, and/or support devices) are provided to job sites inan efficient manner to satisfy the job request needs, such as timing ofjob execution and the like. For example, in some embodiments, the fleetoperations system 12002 may employ “just-in-time” strategies tofacilitate delivery of fleet resources and/or maintenance tasks toensure fleet resources are allocated in an efficient manner withoutsignificantly impacting completion times. In some of these embodiments,the fleet operations system 12002 may leverage the intelligence servicesto anticipate the fleet resource needs corresponding to various jobrequests and/or job execution plans anticipate the fleet resource needsand to arrange for deliver and/or maintenance of such fleet resources.

In some embodiments, job workflows that include multiple dependentstages may be pipelined, such that certain resources are not requireduntil another workflow stage is complete. In such a scenario, the fleetoperating system 12002 may delay the provisioning of the certainresources until the prior workflow stage is nearing completion. In thisway, those resources may be used in connection with another job (oranother part of the same job) while the prior workflow stages arecompleted. In these embodiments, the job execution system 12022 maymonitor the status of certain tasks across multiple jobs to determinewhen the certain resources will be needed. In these embodiments, the jobexecution system 12022 may leverage the platform 12000 intelligencelayer 12004 to predict when tasks will complete. In response, theresource provisioning system 12014 and the logistics system 12015 maywork in combination to provision and deliver the resources to a job sitebefore the previous tasks complete.

In embodiments, the job execution system 12022 may anticipatejob-related resource needs in a job-specific manner to predict whenspecific resources will be required for a specific job. For example, thejob execution system 12022 (working in combination with the intelligencelayer) may generate a schedule of in-progress and/or upcoming tasks fora specific job request, and in response, may determine when certainfleet resources are likely to be needed and/or to come available.Additionally or alternatively, the job execution system 12022 maypredict the job-related resources for a specific job in other suitablemanners. For example, prediction of resource needs may be determinedbased on a pattern of fleet resource needs as derived from a job requesthistory of the job requestor (e.g., a site cleanup job request hastypically followed a completion of a requested job at a job site); aresource usage history of the job requestor from the previous N jobsperformed for the job requestor; timing of job requests (e.g., requestsfrom the requestor are typically received on a Thursday for jobs tostart on Monday the following week), and/or the like. Similarity of ajob requestor to other job requestors (e.g., affiliated entities, directcompetitors, similar SIC codes, and the like) may also form a basis forfleet resource prediction/anticipation. Business relationships amongentities (e.g., a supplier and a shipper, a seller and a buyer, consumerand recycler, and the like) can form a basis for predicting fleetresource needs and timing of the shipper/buyer based on actions,including job requests, of the supplier/seller/consumer.

In embodiments, many other factors may impact fleet resource needpredictions, such as weather forecasting and seasonal affects (e.g.,snow removal and related job requests in northern climates during thewinter season, beach erosion prevention/remediation job requests of warmweather waterfront areas around hurricane season, lawn maintenance jobrequests during the Spring season, leaf cleanup job requests in areaswith deciduous trees in the Autumn season, and the like). Fleet resourceneed prediction may also be activated by events outside of the core jobrequest process, such as natural disasters, vehicleaccidents/emergencies, timing of societal activities (e.g., strandedvehicle support and accident remediation on heavily traveled roadwaysduring rush hour, and the like), scheduled public and/or private events(e.g., cleanup of city streets around a sports venue after completion ofa scheduled match) and the like. In another example, other sources ofinformation that may impact anticipation of fleet resource needs mayinclude business goals and objectives, such as reducing or increasingspending near the end of a financial reporting period (e.g., a fiscalquarter, year, etc.). An indication that a target job requestor intendsto cut back on expenses during the last few weeks or months of a fiscalreporting period may suggest that fleet resources that are typicallyallocated to job requests by the target job requestor will be availablefor other actions, such as maintenance, upgrading, pro-bono work,educational opportunities, fleet promotional activity, allocation toother job requestors and the like. In embodiments, fleet goals orobjectives may also impact fleet resource anticipation and thereforecorresponding preparation activities and the like. One such example is arequired upgrade of a class of robot. In anticipation of needing toreserve the robots in this class, the fleet configuration functions mayallocate alternate robot types that can be reconfigured to satisfy therequirements of the reserved robot class for the duration of the upgradeactivity.

In embodiments, anticipation of fleet resource needs may be determinedthrough use of fleet management platform 12000, such as the platform12000 intelligence layer 12004 and the fleet operations system 12002.For example, in some embodiments the platform 12000 intelligence layer12004 may analyze sources data that may impact fleet resource demands,such as weather forecasts, public activity calendars, job request data(e.g., timing, job parameters, relations to other job requests and thelike), social media postings, government activity/legislation, seasons,and the like. In this example the platform 12000 intelligence layer12004 acting in cooperation with the fleet operations system 12002 maypredict fleet resource demand based on an analysis of the disparate datasources (e.g., using a neural network or the like). In theseembodiments, the platform 12000 intelligence layer 12004 may process thedata from the disparate data sources and determine a likelihood of fleetresource needs across a range of factors.

Other aspects of fleet resource anticipation may include use of the jobrequest process described herein for fleet preparation and/ormaintenance activities, such as by automatically configuring one or morejob requests for fleet preparation-directed activities (e.g.,preparation and/or maintenance of robot operating units or supportingdevices). In this way, the fleet management platform 100 may operate tofacilitate job request performance while ensuring fleet-specific needs(e.g., maintenance) are met. A balance of fleet self-focused activities(e.g., maintenance) with job anticipation needs and further with jobrequests from clients of the platform 12000 may be achieved through useof relative weighting of job requests.

In embodiments, a fleet management platform 12000 may interface withexternal data sources 12036 for performing various platform functionsincluding job configuration, fleet configuration, job negotiation (e.g.,via a smart contract facility), job execution and the like. Examples ofexternal data sources for use by the platform 12000 include value chainentities (e.g., third parties paying for fleet services and the like),enterprise resource planning systems (ERPs) that may provide job contextfor performing team configuration and/or execution of a requested job,smart contracts, and the like. Other external data sources may includethird-party sensor systems (e.g., GPS data, value chain logistics datafor when material needed for a job is to be delivered, and the like) aswell as third-party data streams (e.g., weather, traffic, electricitypricing, and the like).

In some embodiments, the fleet management platform 12000 may support theuse of smart contracts in relation to job requests, job performance,resource, allocation, and/or the like. In embodiments, job requests maybe routed through a smart contract handler that captures jobrequirements, requestor goals and objectives, and fleet job executionconstraints into a dynamic smart contract. In some embodiments, smartcontracts may be utilized throughout a fleet management platform toaddress all manner of fleet operations, such as administering negotiatedrouting of a multi-purpose robot from a first location (e.g., a currentjob site, a warehouse, a temporary storage/service location) to a secondlocation (e.g., a target job site). As a further example, a smartcontract may be put in place as a control for a bidding system for robottime/task utilization. As another example, a smart contract may monitorcertain activities (e.g., task related activities and the like) relatingto a job request. The smart contract may rely on and/or benefit fromaccess to fleet platform data, (e.g., task progress, sensor data, andthe like) to trigger actions defined by the smart contract, such aspayments upon completion of a task or job. The fleet management platform12000 may provide access to fleet resources, including fleet datathrough Application Programming Interfaces, infrastructure elements suchas sensor networks, edge computing systems, and the like for updatingstates relevant to smart contract terms and conditions.

Referring to the embodiments depicted in FIG. 134 , the jobconfiguration system 12018 and the fleet configuration system 12020collectively generate a job execution plan 12310, according to someembodiments of the present disclosure. In embodiments, a job executionplan 12310 may define a set of tasks that are to be performed incompletion of a requested job and may further define a configuration ofa fleet of robot operating units that are to complete the job. Inembodiments, a job execution plan 12310 may include task definitions12304D, workflow definitions 12306D, fleet configurations 12020D (whichmay include robot configurations of individual robots), teamassignments, and references to (or incorporation of) contextualinformation, such as job site details and the like. In embodiments, thejob configuration system 12018 receives a request 12300 that defines thejob to be done and the job configuration system 12018 may determine aset of task definitions 12304D that respectively define a task that isperformed by a robot in completion of a job. In embodiments, the jobconfiguration system 12018 further defines a set of workflow definitions12306D. The workflow definitions 12306D define at least one order inwhich tasks are performed in completion of a project and/or job,including any loops, iterations, triggering conditions, or the like. Inembodiments, the job configuration system 12018 may determine theworkflows 12306D based on the task definitions 12304D that comprise ajob and/or project. The job configuration system 12018 may leveragelibraries of preconfigured workflows to complete certain jobs.Additionally, or alternatively, the job configuration system 12018 mayleverage the platform 12000 intelligence layer 12004 to obtain aninitial workflow definition 12306D for a job and/or project that is partof a larger job. In some embodiments, a human may configure the initialworkflow definition and/or may provide input that is used to determinethe initial workflow definition. In embodiments, the job configurationsystem 12018 may interface with one or more components of the fleetmanagement platform 100 to exchange information for developing a robotfleet job execution plan 12310 and/or to leverage one or more servicesthereof. For example, the job configuration system 12018 may interfacewith the data processing system 12030, a robot configuration library12314 of robot, fleet, project, and task related information, thefleet-level intelligence layer 12004, the fleet configuration system12020, and the like.

In the example of FIG. 135 , the job configuration system 12018 mayinclude a plurality of systems that perform job plan preparationfunctions, by processing the information received in the job request12300. In embodiments, the systems of the job configuration system 12018may include a job parsing system 12302, a task definition system 12304,a workflow definition system 12306, and a workflow simulation system12308. In the illustrated example, the job configuration system 12018systems work in combination to generate a job execution plan 12310 thatis used to define a set of robot operating unit assignments 12312. Inembodiments, robot operating unit assignments 12312 may be supplementalto or integrated with a job execution plan 12310 and may identifyspecific robot teams and/or robots assigned to respective tasks. Forexample, robot operating unit assignment 12312 may define specific tasksand for each task, may identify a specific robot assigned to a task viaa robot unique identifier and/or a specific robot team with a teamidentifier assigned to the task. In embodiments, the robot operatingunit assignments 12312 may be generated by the job configuration system12018 and/or the fleet configuration system 12020.

In embodiments, a job parsing system 12302 receives and parses a jobrequest 12300 to determine a set of job request parameters that areultimately used to determine a job definition, project definition(s),task definitions, workflow definitions, fleet configurations, and robotconfigurations. In embodiments, a job parsing system 12302 may receive ajob request from a user via a user interface, such as the humaninterface system 12024 that receives input by an operator to configure,adapt, or otherwise facilitate parsing of the job request. Additionallyor alternatively, the job parsing system 12302 may receive the jobrequest from a client device associated with a requesting organization.

In embodiments, the job parsing system 12302 may be configured with aningestion facility for receiving electronic versions of job descriptionsand related documents, such as drawings, materials lists, flow charts,GPS data, smart contract data and/or terms, links to the same, and thelike. The ingestion facility may parse documents for keywords,references to activities and the like that can be useful fordetermining, among other things which aspects of the described job maybe suitable for robot tasks. In an example, an ingested document may beprocessed with content and structural filters for detecting portionsthereof for robot automation, such as structural and/or content elements(e.g., indented numbered lists, references to robot identifiers,references to existing robot task content, and the like) that mayfacilitate identification of tasks, sub tasks, sequences of tasks,dependent requirements for tasks, workflow descriptions, and the like.Further keywords in the ingested job content, such as weight terms, jobenvironment terms, and the like may be usefully applied by the jobconfiguration system 12018 elements by providing insight as to thetype(s) of robots needed and the configurations thereof. As an example,a keyword that suggests an object to be moved weighs 14 tons, suggests arobot transport device/team that has at least that amount of movingcapacity.

In embodiments, the job parsing system 12302 may incorporate and/orutilize machine learning functionality (e.g., as may be provided by theplatform 12000 intelligence layer 12004) to improve techniques forparsing job content which may include description data. In addition tomachine-based learning from human-generated feedback on job contentparsing results, learning may be based on experience with other jobcontent parsing actions (e.g., prior job requests), common and specialknowledge bases, such as technical dictionaries, expert humans, and thelike.

In embodiments, job parsing of job content may include automated parsingof structured and unstructured text. In some embodiments, the jobparsing system 12302 may be configured to identify (and optionallyresolving) missing/unclear data and qualified job content data(collectively referred to as “insufficient information”). In response toidentifying insufficient information, the job parsing system 12018 maygenerate and provide a request to a human operator via a user interfacefor clarification with respect to the insufficient information. Such arequest may identify specific inputs from the user to provide, such thatthe request identifies the clarifying content that was missing orunclear initially. Additionally or alternatively, the parsing system12302 may determine the clarifying content from (e.g., through a queryof) a library 12314 that maintains data from prior job requests, suchthat the clarifying content may be obtained using the prior job requestinformation and context from the request. If the parsing job is unableto determine the clarifying content, the parsing system 12302 maygenerate a request for clarifying content, as discussed above.

In embodiments, a range of job description information may be providedto, determined, and/or extracted by the job configuration system 12018.Examples of job request parameters may include, but are not limited to:(i) physical location information that could be used to automaticallydetermine transportation options, operational restrictions, permitting,travel restrictions, local assets, logistics, etc.; (ii) available sitepower voltage, frequency, current, etc. may restrict availableequipment, or require additional equipment, especially for support;(iii) digital data for a site layout, such as 3D CAD models, scans,robot surveys may be available or might be completed as part of initialproject scoping, and may be used to automatically provide task priorityand workflow routing, robot selection, supervisory needs, etc.; (iv)operating environment including temperature, hazard description(s),terrain, weather, etc.; (v) deliverables, such as data, reports,analysis, and the like; (vi) customer interfaces for data exchange, suchas network interfaces, APIs, security; (vii) communication networkavailability, such as land line, 4G, 5G, WiFi, private networks,satellite, connectivity constraints, and the like; (viii) budgetconstraints for equipment limitations, time on site, permitting; (ix)scheduling for site availability, reconfiguration flexibility, earlieststart time, latest finish time, rate of activity, such as the number ofrobots active at any given time, and the like. Examples of other jobdescription information that may be handled by a job parsing system mayinclude contract-related information, such as smart contract terms,certification level of robot operational software for robots deployed onthe job site, insurance provisions, site access requirements (e.g., ajob site can be accessed only when humans are not present or onlythrough coordination with humans that are present on the site),conditions for assigning a proxy for a task, activity, workflow or theentire job.

In embodiments, the job configuration system 12018 systems (e.g., jobparsing system 12302, task definition system 12304, workflow definitionsystem 12306) may reference a library 12314 to identify content andstructural filters for distinguishing robot automation job content fromother job content (e.g., cost, payment, financing, etc.), preconfiguredcandidate tasks, workflows, and/or complete job configurations thatsubstantially meet the requirements of the job request. In embodiments,the library 12314 or another job configuration library may facilitatemapping indicia of the job content with target terms that indicate robotautomation. As an example use of an automated task from the library12314, a requested data collection job may include a requirement forsampling surface water in a storm system catch basin. The job parsingsystem 12302 may identify the sampling requirement, and in response thetask definition system 12304 may identify an automated sampling task forsampling water in the library 12314 that meets the requirements of thatportion of the job request description, which may be used in definingthe job execution plan 12310. If job configuration system 12018determines that a suitable job configuration is available (e.g., fromthe library 12314), such as if the job requested had previously beenrequested, the job configuration system 12018 may use a previous jobexecution plan 12310 corresponding to the previously requested job as aproposed job execution plan 12310 for further validation with currentfleet standards and the like. For example, the platform intelligencelayer 12004 may analyze the proposed job configuration (e.g., with oneor more intelligence services, including without limitation a machinelearning service) with respect to a set of governance standards toensure that the proposed job configuration comports with said standards.The platform intelligence layer 12004 may perform otherintelligence-based tasks with respect to the proposed job configuration.

In some scenarios, the job configuration system 12018 may determine thatone or more tasks, workflow, routines, and the like do not have asuitable counterpart in the library 12314. In such a scenario, the jobparsing system 12302 may generate a data set that includes robot-fleetfocused requirements (e.g., task definition parameters, robotconfiguration parameters, suggested task order, and the like) forperforming the task that is passed along to other job configurationsystem modules for processing. In embodiments, the job parsing system12302 may rely on the platform 12000 intelligence layer 12004 forsuggestions of such requirements, including combinations of tasks thatwhen optionally adapted may satisfy the job requirement. In an example,a job requirement may include sampling surface water from a frozen stormcatch basin. In this example, the library 12314 may not include a frozensurface water sampling task. However, the platform 12000 intelligencelayer 12004 may recommend an ice melting task followed by a watersampling task to meet the job requirements.

In embodiments, the job parsing system 12302 may include and/orinterface with the analysis modules/governance libraries of theintelligence layer 12004 of the platform 12000. The job parsing system12302 may leverage the governance-based analyses by providing portionsof the candidate robot automation portions of the job content (e.g.,terms and the like) for processing. The intelligence layer 12004 may, inresponse to the provided portion of job content, provide and/or indicateone or more of safety standards and/or one or more of operationalstandards to be applied during preparation of the job execution plan bythe job configuration system 120118.

In embodiments, the job parsing system 12302 may include a jobrequirements module that produces a set of job request instance-specificrequirements for use when the job configuration system 120118 definesrobot tasks, configures fleet resources, define workflows, simulatesworkflows, generates a job execution plan, and/or the like. Inembodiments, the set of job request instance-specific requirements maybe determined based on at least one or more of: (i) the candidateportions of the job content that indicate robot automation (e.g., termsthat indicate a robot task), (ii) one or more inputs from the userinterface (e.g., clarification of terms), (iii) safety and operationalstandards (e.g., from the governance layer), and (iv) a recommendedrobot task and associated contextual information (e.g., provided by afleet intelligence layer).

In embodiments, the job content parsing system 12302 may apply contentfilters and/or structural filters to identify structural elements in thejob content that may indicate one or more of tasks, sub-tasks, taskordering, task dependencies, task requirements and the like. Inembodiments, the detected structural elements may facilitate selectionand configuration of robot operating units by, for example, the fleetconfiguration system 12020. In an example, a structural element thatdistinguishes set of tasks may be used by the fleet configuration systemto avoid assigning the same robot operating unit to tasks within the setof tasks delineated by the structural element and tasks outside of theset.

In embodiments, the job parsing system 12302 may incorporate and/orutilize a job request configuration agent/expert system that may beconstructed to facilitate developing job description parsingcapabilities.

In embodiments, the task definition system 12304 may organize job datainto task definitions 12304D (e.g., discrete robot tasks or tasksperformed by robot teams). The task definition system 12304 may furthercoordinate other systems of the job configuration system 12018, such asthe workflow simulation system 12308 to optimize the task definitions.

In embodiments, the task definition system 12304 may refine job datacompiled by the job parsing system 12302 to facilitate defining discreteoperations of one or more robot operating units in the fleet of robotsin performance of a requested job. Defining tasks may be based oninformation regarding robots, robot types, robot features, and robotconfigurations that can perform a defined task. In embodiments, the taskdefinition system 12304 may further provide information in taskdefinitions 12304D that facilitate a fleet configuration system 12020 indetermining use of general/multi-purpose robots, special purpose robotsand/or combinations thereof for each defined task. In embodiments, thetask definition system 12304 may define tasks that meet a first fleetobject of a set of fleet objectives. A first fleet object may includedefining tasks that can be performed by a multi-purpose robot by, forexample, breaking down job content into smaller tasks that require lesscustomization of the robot. In embodiments, the task definition system12304 may reference the library 12314, the platform 12000 intelligencelayer 12004, or other platform-specific or accessible resources whenmaking task suggestions.

As the task definition system 12304D defines the tasks of a job, thetask definition may be cataloged and stored for future use, such as inthe library 12314. In some embodiments, the task definition system12304D may adapt a task definition from a previously cataloged taskdefinition (e.g., adapting a task definition for a particular type ofenvironment or certain conditions thereof from a previously cataloguedtask definition). In these embodiments, the task definition system12304D may catalogue the derivative task definition in the library 12314with adaptation instructions. In some embodiments, a task definitionthat is catalogued in the library 12314 may be associated with analready cataloged task definition and/or may replace an alreadycataloged task definition, may be cataloged as a sub-task of an existingtask and the like. In general, task definition may include associatedtasks, serialized tasks, nested tasks and the like.

Information about a job may be stored in the library 12314 for futureuse, therefore, the task definition system 12304 may access the library12314 to retrieve information about the job, robots, fleets, and thelike. In the current exemplary embodiment of inspection of a ventilationsystem, the information accessible through the library 12314 mayinclude, for example how to access information about the physicalconfiguration of the ventilation system. The task definition system12304 may also access the library 12314 to update information, such asby adding one or more tasks to a list of tasks for the ventilationinspection job, results from optimizations of task definition performedby the job execution system, and the like.

Optimization features of the task definition system are described belowin association with feedback from other elements of the jobconfiguration system 12018, such as the workflow simulation system 12308and the like.

Task definitions may be generated and provided to other elements of thejob configuration system 12018, such as the workflow definition system12306 and a fleet configuration system proxy 12305. In embodiments, thefleet that may provide the task definitions (and other suitableinformation) to the fleet configuration system 12020. In an example, afleet configuration system proxy 12305 may narrow down sets of candidaterobots for performing tasks (as indicated in task description(s) 12304A)to a specific robot type (and optionally a specific robot in the fleet)based on fleet configuration and fleet resource inventory and allocationdata relevant to the requested job (e.g., based on geography, timing,and the like). The fleet configuration system proxy 12305 may processtask definitions, which may include robot identification information(e.g., robot type and the like), for aligning resources of the fleetwith the relevant task information. In an example, a fleet configurationproxy 12305 may generate data suitable for use by fleet operationalelements, such as a fleet resource provisioning system 12014, to performfleet resource allocation, scheduling, and the like that supports atleast a portion of the goals of a job request being processed throughthe job configuration system 12018. The fleet configuration proxy 12305may employ fleet configuration modeling to determine candidate fleetconfigurations that meet job requirements. The modeling may be useful indetermining an impact on fleet resources that may then be taken intoconsideration during fleet configuration functions, resource allocation,and the like. In embodiments, fleet configuration modeling may includeuse of platform intelligence layer resources, such as machine learning,artificial intelligence, and the like when determining one or morepreferred fleet configurations that also satisfy one or more jobdescription requirements. The fleet configuration system 12020 isdescribed in further detail elsewhere in this disclosure.

Workflow Definition System

In embodiments, the job configuration system 12018 may include theworkflow definition system 12306 that receives task definitions from thetask definition system 12304, fleet configuration information from thefleet configuration system 12020, other job request information that mayfacilitate task sequencing (e.g., timing of deliverables and/or tasks)and generates one or more task workflows based thereon. In embodiments,the workflow definition system 12306 incorporates information from thefleet management system to identify workflow possibilities using outputfrom the task definition, job parsing system, and real-time externaldata such as maintenance management systems, ERP systems, and so forthto determine the task workflows. In embodiments, a task workflow definesan order and manner in which tasks are performed for performing aproject/job. In embodiments, the workflow definition system 12306 mayapply job descriptive information to a set of task definitions and fleetconfiguration data to produce one or more workflows to perform one ormore activities of the job. As an example, a workflow may cover anactivity such as entering a ventilation conduit via a portal, such as aventilation inlet port and the like. The tasks defined for this activitymay be collected into a workflow or portion thereof, ordered to ensureproper compliance with the job requirements, and published as a set ofrequirements to perform the activity/workflow. A job workflow definitionmay include information descriptive of quantities and types of robots,tools/end effectors, and the like that may be provided by the fleetconfiguration system 12020 for one or more tasks being ordered by theworkflow definition system 12306. In embodiments, this portion of theworkflow definition may be utilized by other modules of the jobconfiguration system 12018 (e.g., job execution system 12022) to, forexample, identify and determine required configurations of one or morerobots, and the like to be readied ahead of performing a task in theworkflow (e.g., ensuring that a multi-purpose robot is (re)configuredwith a configuration that enables performing a task prior to performingthe task that is defined in the workflow). Other information produced ina job execution plan may include sequence of tasks (e.g., as produced bya workflow system), which may further identify a sequence of robotsrequired to perform the tasks.

A workflow definition system may utilize resources of the robotconfiguration library 12314 when defining workflows. Workflow definitionparameters, such as how to determine minimum time between tasks,inter-task coordination, task classification, workflow scope and thelike may be available in the library 12314, and/or in informationretrieved from a job request. These and other parameters may includejob-specific variables that can be set to default values, but adjustedby, for example, the workflow definition system to meet job-specificneeds. An example of use of robot configuration library 12314information to develop job workflow definitions may include a robotmovement task followed by a sampling task. Information in the robotconfiguration library 12314 related to the material/object to be sampledmay indicate that a minimum dwell time after the robot is dispositionedmust be satisfied before the sample, such as to allow ambient dust tosettle, and the like. Other useful information that a workflowdefinition system may utilize from a robot configuration library 12314may include template, preconfigured or default workflows, such asworkflows developed for a previous execution of the job. A workflowdefinition system may determine which, if any, workflow in the library12314 (base workflow) is suitable for use in the current job workflowdefinition instance; determine adjustments to the retrieved workflow;and produce an instance-specific job workflow that may includeadditional tasks not found in the base workflow and/or excludeunnecessary tasks found in the base workflow, and the like.

Other examples of robot configuration library 12314 information that maybe useful for to develop job workflow definitions include availabilityof sensor detection packages. These sensor detection packages mayindicate a preferred sequence of sensing tasks and therefore may impactworkflows of such tasks. These and related reconfigured sensor anddetection packages may combine sensor selection, sensing, informationcollection, preprocessing, routing, consolidation, processing, and thelike. These sensor and detection packages may be included in a fleetconfiguration process, such as being included in a job execution planfor use by the job execution, monitoring, and reporting system 12022. Inembodiments, use thereof is indicated as serving a range of monitoringactivities and the like.

A job workflow definition system may examine task to task dependency(e.g., performing a second task is dependent on completing a first task)to identify potential workflow independence and dependence for amongother things configuring a job execution plan that may includeparallelized use of fleet resources, such as teams and the like.

Features of an intelligence layer, such as the team twin capability,fleet twin capability, and the like may also be beneficially applied tosimulate and validate workflows, such as with the workflow simulationsystem 12308 of the job configuration system 12018. The workflowsimulation system 12308 may perform simulations of portions of a jobconfiguration, such as those portions organized into job workflows bythe workflow definition system. In an example of workflow simulation, aset of tasks defined by the task definition system and organized into aportion of a job workflow may be modeled using functional equivalentsfor robots, tasks, workflows and the like, such as robot twins, tasktwins, workflow twins, team twins, and fleet twins. These twins may beretrieved from the library 12314 and executed by a processor to simulatethe set of tasks, such as to validate the defined tasks. In embodiments,the fleet intelligence system may be utilized for providing at least aportion of these workflow simulations, such as by applying workflowsdefinitions and task definitions to one or more workflow models and/ortask/robot/fleet twins operating in an artificial intelligenceenvironment machine learning environment.

The workflow simulation system 12308 may also generate feedback fromsimulating workflows defined by the workflow definition system that maybe useful in improving a workflow definition, a task definition, a robotselection and the like.

The workflow simulation system 12308 may establish or otherwise accesscriteria for determining if a workflow meets the criteria, such astimely and successfully completing a task, job, and the like. Byapplying these criteria for measuring outcomes of workflow simulations,the workflow simulation system 12308 may validate one or more workflowoptions, robot options passed along to the workflow definition system,fleet configuration options, and the like before providing feedback to,for example the task definition system, the job parsing system and thelike. Options that do not meet the criteria (e.g., consumes an excess ofresources, results in wear down of a robot, fails to meet a schedule andthe like) may be marked as such for improving job configurationfunctions, such as structuring tasks into workflows and the like.

Further the workflow simulation system 12308 may leverage the platform12000 intelligence layer. In embodiments, the platform 12000intelligence layer may provide access to and operation of instances offleet twin modules that may provide critical understanding offleet-based impacts on workflow definition for performing a requestedjob. In embodiments, a logistics twin of the fleet intelligence systemmay provide useful workflow simulation information through operation ofmodeling of shipments and costs of robots, personnel, support equipmentand the like for robot fleet delivery to a job site. This modeling offleet logistics may reveal that a local fleet that will soon becomeavailable (perhaps after the preferred start date of a requested job)may complete the job at a lower cost than using a currently availablecrew that requires logics and transportation to the job site. Inembodiments, a fleet twin may facilitate identifying robot operationalassets that are available during the scheduled job by modeling fleetoperations, such as robot maintenance requirements for robots during thepreferred job execution time. In embodiments, a task twin capability ofthe fleet intelligence system may facilitate modeling of robotconfigurations, such as when a multi-purpose robot is reconfiguredduring a job (e.g., during a task) to perform different tasks (e.g., (i)bringing a ventilation inspection wand to a ventilation system port; and(ii) collecting and dispositioning debris being removed from theventilation system. A task twin capability of the fleet intelligencesystem may further benefit workflow definition clarity through workflowsimulation by applying a virtual set of preconfigured robot twins toperform a candidate workflow, or portion thereof, that is optionallybeing defined. In embodiments, a team twin capability of a fleetintelligence system may benefit a workflow simulation system of the jobconfiguration system 12018 by using, for example, preconfigured robotteams to operate and validate candidate workflows prepared by theworkflow definition system.

In embodiments, a result of workflow simulation may include one or moredata structures that are suitable for use in a job execution plan.

In addition to task definitions, robot definitions, workflowdefinitions, fleet configuration parameters, and the like, a jobexecution plan may identify contracts for the job, such as smartcontracts that may be constructed/configured by or in association withthe job configuration system 12018, delivery times for job resources(e.g., fleets of robots), a schedule of deliverables, and the like.

In embodiments, the fleet configuration system 12020 configuresresources of a fleet for a job based on the task definitions and/orworkflow definitions. The fleet configuration system 12020 may determinethe fleet configuration based on other considerations, such as budget,environmental conditions, time constraints, available inventory ofrobots and/or parts, and/or the like. The fleet configuration system12020 may operate cooperatively with a job configuration system 12018,such as when tasks are to be organized into workflows. Task definitionsmay, for example, define tasks that can be performed by special purposeor multi-purpose robots. Job workflows may be impacted by availabilityof each type of robot, so a job configuration system 12018 may leveragethe fleet configuration system 12020 when determining candidate jobworkflows. As an example, a workflow that includes allocation by thefleet configuration system of a special purpose robot (e.g., the specialpurpose robot can be provided for the job being configured) may need tobe adjusted (as compared to the workflow utilizing a multi-purposerobot) to account for differences between these types of robots. Thespecial purpose robot may perform a task or tasks more efficientlyand/or with greater precision than a multi-purpose robot; therefore, aspecial purpose robot workflow may be configured with a shortercompletion time (e.g., greater robot efficiency) or without anindependent confirmation step (e.g., greater precision orself-validating special purpose robot capability). These are merelyexamples to illustrate the potential for impacts on workflow definitionof a fleet configuration system.

In embodiments, fleet configuration for a requested job may includeconfiguring fleet resources into a robot team that is assigned to aspecific task and/or project (noting that a robot or a team of robotsmay be assigned multiple tasks and/or projects). Each robot team mayinclude one or more robot operating units, which may comprise any one ormore of special purpose robots, multi-purpose robots, rigid and/or softrobots, exoskeleton robots, humans, work animals, and the like. Further,a configured robot team may be job-specific and team membership may betransient for any given robot operating unit. As an example, a specialpurpose welding robot, or optionally a multi-purpose robot configured toperform welding operations may be assigned to a first robot team foronly the duration of time during which welding operations are beingperformed by the first robot team. The same welding-capable robot mayalso be assigned to a second robot team for only the duration of timeduring which second robot team welding is being performed. Time sharingof fleet resources, such as a welding-capable robot can be communicatedto a job configuration system from the fleet configuration system 12020,for example, so that workflows being defined by the job configurationsystem can consider availability of the welding-capable robot for eachof the robot teams. In embodiments, any given robot or group of robotsmay be assigned to multiple teams spread across multiple jobs by thefleet configuration system 12020 using a robot-specific time-sharingapproach or other resource utilization optimization technique. In anexample, a fleet configuration system 12020 may use a multi-dimensionalrobot utilization planning system that allocates each robot in a fleetto one job during a unit of time, such as a day, hour, or fractionthereof, allowing each instance of a job configuration system to requestuse of the robot for a specific time (e.g., Tuesday the 23rd from 10AM-4 PM) or a quantity of time units (e.g., six consecutive hours). Thefleet configuration system 12020 may respond to the request with robotfleet configuration descriptions that inform job workflow definitionsand the like.

In embodiments, a fleet configuration may further include multi-purposerobot configuration information (e.g., as may be indicated by a taskdefinition system and the like) for configuring multi-purpose robotsthat are included in a team or fleet of robots for performance of one ormore tasks in a job. The multi-purpose robot configuration informationmay define modules that may be coupled to the robot, including endeffectors, motive adaptors, sensors, image processing modules,special-purpose processing modules, communications modules, and/or thelike. Multi-purpose robot modules and their utilization are furtherdescribed elsewhere herein.

In some embodiments, fleet configuration for a requested job may includeallocating robot support resources, such as edge devices, chargingcapabilities, local data storage capabilities, shipping containers,docking stations, spare parts, required technicians, and the like. Inembodiments, the fleet configuration system may also assign robots todistinct roles, such as roles related to team organization (e.g.,supervisor), security, human interaction, inspection/quality control,and the like. These roles may not be separately defined in a jobrequest; however, criteria in a job request (e.g., quality inspectionreporting) may lead to such robot role assignment. In embodiments, thefleet configuration system 12020 may designate some team roles forhumans, including human team member participation requirements, support,equipment, and the like. A fleet configuration system may take intoconsideration human safety when designating a human as a team member. Asan example, a human team member may be required to wear a safety faceshield when participating on a team that is performing weldingoperations.

In embodiments, the fleet configuration system 12020 may leveragelibraries to determine the fleet configurations. In these embodiments,the fleet configuration system 12020 may determine team configurationsfor defined tasks or projects using a library 12314 that definesdifferent configurations to perform certain tasks, whereby a lookuptable or other association is used to determine the team configurationsfor given a set of tasks. In embodiments, the library 12314 may includeattributes of different robot types, such as a multi-purpose robot. Asan example, an attribute of a multi-purpose robot may indicate a minimumsize of a multi-purpose robot. In embodiments, the fleet configurationsystem 12020 may filter the types of robots that may perform a taskbased on the attributes and one or more job request parametersidentified by the job parsing system 12302 (and optionally configuredinto a task definition). When a task or job operation requires (e.g.,based on data generated by the job parsing system 12302, an existing jobexecution plan 12310, a job request 12300, and the like) access to aspace that is smaller than the minimum size multi-purpose robotavailable, the fleet configuration system 12020 would not include themulti-purpose robot; instead it would attempt to identify a differentrobot and/or robot type/configuration that could meet the sizerequirements. In embodiments, a fleet configuration system 12020 mayreference combinations of robot sizes/types and the like to fitrequirements of a defined task. Further the fleet configuration system12020 may suggest two robots to perform a task when one may not meetother requirements of the task. In a simple example, a task thatinvolved traveling a long distance and then performing an action in asmall space might be resolved by the fleet configuration system with acombination of robots, such as a multi-purpose robot that travels longdistances efficiently (and optionally includes a payload carryingcapability suitable for transporting a special purpose robot) and aspecial purpose robot that meets a small space requirement. Inembodiments, the fleet configuration system 12020 may deliver to the jobconfiguration system 12018 fleet definitions that include a plurality ofrobots, robot types, robot configurations, and the like. A general goalof a fleet configuration system 12020 may include generating fleetconfiguration(s) that require the fewest robots and/or robot types forproper execution of a portion of the requested job. However, the fleetconfiguration system 12020 may work cooperatively with the taskdefinition system 12304 to generate a task-specific fleet configurationthat includes more than one robot type/configuration/combination therebyallowing other elements of the fleet management system 12000 toefficiently manage execution of a requested job. Such a fleetconfiguration may indicate a preferred robot and/or robot combinationfor meeting a goal, such as efficient use of robots and the like thatother elements of the job configuration system (e.g., a job workflowgeneration system) may consider when configuring, for example, aplurality of defined robot tasks into a job workflow 12306D. Therefore,a fleet configuration may include first, second, and tertiary robotindications for performing a task. Alternatively, a fleet configurationfor a job request may identify a plurality of robots, each assignedutilization weights based on criteria, such as efficient job completion,profitability, fleet robot use preferences and the like.

In embodiments, the fleet configuration system 12020 may reference aninventory data store to determine the available robots and/or modules(e.g., physical modules and/or software modules) to configure amulti-purpose robot, locations of those robots and/or parts, statuses ofthe parts (e.g., whether maintenance is due or needed for availablerobots or parts), and the like. In this way, the fleet configuration fora job, task, team or the like may be determined by the availableinventory of robots, modules, support equipment, and/or spare parts.Further, a fleet maintenance management system as described herein maytrack aspects of robot status that may be added to and/or besupplemental to the inventory data store, such as which robots are beingreserved from use for critical maintenance, which robots can bedeployed, but with diminished capability due to service and/ormaintenance or other concerns, status of spare parts or other serviceactivities (e.g., due date, current location, anticipated installation,and the like). Therefore the fleet configuration system 12020 mayreference and/or be informed by the fleet maintenance management systemabout fleet resource maintenance knowledge that may be job-impacting.Additionally, or alternatively, the fleet configuration system 12020 mayrequest a fleet configuration from the platform 12000 intelligence layer12004, where an artificial intelligence service 12028 may receive a setof parameters, including task definitions, workflow definitions, budget,environment definition, job timeline, or the like as input, evaluate aplurality of candidate fleet configurations and determine a target fleetconfiguration that can perform the job. In embodiments, a human candefine or redefine any portion of a fleet configuration via a humaninterface of the fleet configuration system.

In embodiments, the job and fleet configurations may be fed to a digitaltwin system, whereby the digital twin system may perform a simulation ofthe job given the job and fleet configurations. The job configurationsystem 12018 and/or the fleet configuration system 12020 may iterativelyredefine the job configuration and the fleet configuration to optimize(or substantially optimize) one or more parameters, such as a jobtimeline, overall cost, robot downtime, maintenance-related downtime,shipping costs, or the like. Once the job configuration system 12018 andthe fleet configuration system 12020 have determined the task andworkflow definitions, as well as the fleet configurations, includingmulti-purpose robot configurations and team assignments, the fleetmanagement platform may output the job execution plan 12310corresponding to the job request.

In embodiments, the fleet configuration system 12020 may leveragedigital twins when configuring fleet resources. Use of digital twinswith fleet configuration may include identifying and/or defining one ormore digital twins of one or more robots based on information in thetask definition 12304D. Fleet configuration may include identifyingconfiguration and/or operation of a multi-purpose robot so that amulti-purpose robot can perform the task or a portion thereof. Suchmulti-purpose (and optionally special purpose) robot task configurationinstructions may be generated through the use of a digital twin for oneor more of a set of candidate robots for performing a task. In anillustrative example, a multi-purpose robot may be associated with aplurality of configuration/operational data structures for configuringthe multi-purpose robot to perform routines, actions, tasks and thelike. The fleet configuration system 12020 may identify or otherwise beprovided with one or more candidate multi-purpose robot configurationdata structures (e.g., from the library 12314) for use to perform atask. A portion of such a candidate configuration data structure mayinclude a rotational rate for an end effector to secure a panel rotatingretention bolt. The requested job requirements may explicitly orimplicitly indicate that a rotational rate for securing a panel isdifferent than the value in the candidate configuration data structure.In embodiments, the fleet configuration system may make any adjustmentsto the candidate configuration data structure (e.g., reducing rotationrate), apply it to an instantiation of a digital twin of the candidatemulti-purpose robot, observe and/or evaluate the execution (e.g.,simulation) of the digital twin with the adjusted configuration datastructure, and store it in the library 12314 and the like. The newlystored configuration data structure may be cataloged based on the jobrequest and/or other parameters of the requested job, task, and the liketo make for efficient access in the future.

A robot configuration library 12314 may include job information, robotinformation, fleet information, task definition rules/metadata that maybe useful to determine how to define robot tasks, workflow configurationrules and/or techniques, prior job request results from application ofthe job configuration system (e.g., prior job execution plans), and thelike. This library 12314 may be accessed and/or updated by functions ofthe job operations platform. Illustrative examples of the library 12314are described herein variously in conjunction with job operationsplatform functions and features, such as job configuration and the like.As an example, the robot configuration library 12314 may includespecific reference to configurations of multipurpose robots that may beutilized during fleet configuration, job execution and the like. In thisexample, the robot configuration library 12314 may have references torobot configuration data sets (e.g., data that when uploaded to amultipurpose robot may enable the robot to perform a function, such asstanding, welding, and the like). Further the library may provide across-reference of multipurpose robot configurations with otherrobot-related information, such as base model, version, requiredfeatures, and the like that may be required for successfully deploymentof a robot configured with a given configuration. Yet further, thelibrary may suggest alternatives to certain combinations of robot andconfiguration, such as indicating that a newer version of a robot modelmay include built-in capabilities provided by a specific configuration.Therefore, the fleet configuration system may have greater flexibilityin deciding which robots to deploy for different jobs. References aremade herein to the library 12314, using contextual modifiers, such asrobot configuration library and the like. These contextual modifiers maysuggest one or more portions and/or instance of the library 12314 forillustrative purposes only.

In embodiments, optimization features of the task definition system aredescribed below in association with feedback from other elements of thejob configuration system 12018, such as the workflow simulation system12308 and the like.

FIG. 136 presents a flow diagram showing an embodiment of the fleetoperations system and a data flow thereof. In the example embodiments,the fleet operations system and the fleet intelligence system perform afeedback for job execution-time iteration of configuration activities,such as for adapting an executing instance of a job execution plan. Theembodiments of FIG. 136 depict an embodiment of the methods and systemsof a robot fleet platform 12002 depicted and described herein, in whichfeedback within a job configuration system 12018 facilitates iteratingconfiguration activities when producing components of a job executionplan 12310, such as task definitions 12304D and workflow definitions12306D. As described for these embodiments, the fleet intelligence layer12004 may be used for at least these iterations. However, it isenvisioned that the resources of the fleet intelligence layer 12004 mayalso or in addition be used for enhancing execution of a job executionplan 12310.

In the example of FIG. 136 , the job execution system 12022 of the fleetoperations system 12002 may receive job execution plans 12310 from thejob configuration system 12018 responsive to, for example a job request.The job execution system 12022 may facilitate performance of a jobexecution plan 12310 by stepping through the plan, activating andmonitoring robot units and other fleet resources, and providing feedback12322, optionally real-time feedback based on, for example, robot unitmonitoring data. This feedback 12322 may be processed by, for example,artificial intelligence capabilities of the fleet intelligence layer12004 for determining adjustments to a job execution plan, such as taskdefinitions and the like. When this feedback and adjustments are done inreal-time or near real-time (e.g., before an upcoming job executionactivity, such as a step in a workflow 12306D), functions of the jobconfiguration system 12018 may be iterated to amend an existing jobexecution plan, such as an instance of a plan that is currently beingexecuted by the job execution system 12022. In a building ventilationinspection example of job execution plan iteration, a task of entering aventilation system may involve removing a ventilation portal cover at aplurality of locations in the building. Based on job execution-timefeedback from a robot (or team of robots) removing the initialventilation portal cover from a ceiling port, the definition of thistask may be adapted to require a different retention technique forholding the cover in place without damaging it while removing thefasteners. In embodiments, the feedback may include images and/or videoof the removal task. In embodiments, the feedback may include ameasurement of the weight of the cover as determined by the robot(s)performing the removal task.

This real-time (or near-real time) visual feedback may be analyzed bythe fleet intelligence system to determine, for example, that a portionof the baffles on the cover were deformed during removal. An artificialintelligence system of the fleet intelligence layer 12004 may performsimulations of various cover support techniques and recommend one ormore as input to the job configuration system 12018 for updatingcorresponding task definitions. In embodiments, the fleet intelligencesystem may send an alert to the fleet operations system 12002 regardingthe need for adapting this task definition that may be used by thesystem to update, for example, preconfigured task definitions stored inthe robot task library 12314 and the like. Such an alert may be used bythe fleet operations system to coordinate with the job execution system12022 so that pending ceiling-based ventilation cover removal tasks arenot executed before being refreshed in the job execution plan 12310. Inembodiments, the job configuration system 12018 may release onlyportions of the job execution plan 12310 to the job execution system12022 so that unreleased portions can be adapted; thereby mitigatingimpacts on the job execution system, such as requiring work to behalted, delayed, or otherwise impaired while updates to the executionplan are made.

While the examples for job configuration and the like presented hereingenerally consider a single job being configured by the jobconfiguration system 12018, there may be many jobs being configuredconcurrently. The methods and systems for real-time or near real-timefeedback described herein may apply to any instance of job configurationactivity being performed so that feedback on task definition of a firstjob may benefit task definition of a second job, while maintainingnecessary job-isolation requirements (e.g., job identifying data may beobfuscated) to support concurrently processing job requests fromdifferent entities.

Also, depicted in FIG. 136 is a means for further enhancingconfiguration activities (e.g., job and fleet configuration as describedherein) for handling future job requests by optionally capturing datarepresentative of completion of a requested job as a form of feedbackfor use by the fleet intelligence layer 12004 for, among other things,learning and optimization. In embodiments, capturing data representativeof completion of a requested job may include extracting such data from ajob completion data set 12326. This job completion data set 12326 may beconstructed to facilitate identifying information that may be useful forlearning and optimization 12324. In an example, the job completion dataset my designate, such as by use of metadata tags, logical and/orphysical separation, or other indicia data that represents exceptions orlarge variants from expectation. In an example, at job completion, acount of repetitions of a robot function (e.g., articulated armmovements to remove debris from a building ventilation system) mayexceed an expected number. This excessive count of repetitions may beflagged as candidate information for learning and optimization feedback12324 to be extracted and sent to the fleet intelligence layer 12004. Inembodiments, a job execution plan 12310 may be configured withindicators of types of data to be collected and used for learning andoptimization feedback 12324. The fleet intelligence layer 12004 mayrecommend to the job configuration system 12018 the types of data to beso indicated based on other factors known to the fleet intelligencesystem, such as inquiries made by robot design engineering teams and thelike. In embodiments, learning and optimization feedback 12324 may beused by the fleet intelligence layer to perform, among other things,optimization of artificial intelligence service (e.g., recommendingrobot teams, robot types, workflows, and the like). Referring todescriptions herein, preconfigured tasks, robot configurations, teamconfigurations, and the like may be retrieved from the library 12314.When these preconfigured aspects of a job execution plan are executed,data representative of the performance thereof may be flagged for use aslearning and optimization feedback 12324 to continuously improve thesepreconfigured aspects. An outcome of use of this data includes fieldcondition-adapted preconfigured tasks b that may perform better in thereal world. Another outcome of use of this data includes improveddigital twins and machine learning models.

Referring to FIG. 137 , embodiments of a job parsing system 12302 and atask definition system 12304 are depicted in an interconnected block anddata flow diagram. A job description to be parsed may include relevantjob descriptive details, goals, objectives, requirements, preferencesand the like and as may be described elsewhere herein. While not allpertinent job information may be included within the request, one ormore links to ancillary job description data 12404 may be included.Ancillary job data 12404 may be stored remote from a job request dataset (e.g., may be accessed through an Internet URL of the jobdescription). Optionally, ancillary job data 12404 may be stored in datastructures that are accessible to the fleet management platform 12000,such as in a fleet library 12314, requestor-specific storage, and thelike. Ancillary job data 12404 may include formal standards (e.g., localdisturbance regulations, safety (OSHA), electrical (NEC), quality, andthe like), permitting requirements (e.g., forms, steps, timing,dependencies on other tasks, and the like), legal requirements (e.g.,union approval, relevant laws, and the like) details of the job,requestor work standards (e.g., a workmanship standard for therequestor), industry norms (e.g., work hours, material selection,templates, and the like) approved vendors (e.g., from whom supplies andother consumables are to be acquired), references to preconfiguredtasks, user interface templates/menus/screen for each aspect of a job(e.g., how a user can request status, observe activity, change a jobrequirement, respond to an inquiry, and the like) and the like. The jobrequest data and, if indicated, the ancillary data 12404 are processedby a task definition ingestion facility 12402 that works cooperativelywith a job data conversion module 12403 to generate jobinstance-specific content 12408. This job instance-specific content mayinclude, among other things, initial sequence timing as may be definedin the input data (e.g., “do task A before task B”) and/or derivedtherefrom (e.g., installing an object necessarily must occur after theobject is received). The job data conversion module 12403 may interactwith the data processing system 12030 when converting job descriptiondata to utilize information derived from a fleet management platformaccessible library, such as job and fleet library 12314. The ingestionfacility 12402 may store some job description content directly into thejob instance storage 12408, such as job identification information,links to internal ancillary data and the like.

In embodiments, one or more human interactive capabilities forfacilitating job parsing and task definition may include knowledge-basedsystems (e.g., AI-based and the like) that may interact with a human(e.g., via text input, conversation-bot, haptic-input, and the like) togather information for preformatting, organizing, and vetting job andtask data. These interactions may be in lieu of or supplemental toreceiving a job description. As an example, a job description mayinclude a reference to performing tasks after normal work hours, whichmay include working after sundown. The interactive job descriptionvetting capabilities and others mentioned here, may determine thatclarification could benefit job description parsing and task definition,such as will the job require human-suitable illumination and if so underwhat conditions. Because robot sensing may not require such illumination(e.g., robot visual functions may be met through use of infrared orother non-human visible light emissions), human-visible lighting mayonly be required to be deployed at certain times during job execution(e.g., at start of a function, when a delivery is being made, when ahuman inspector is on-site, and the like). By providing a capability forhuman interaction as part of job parsing, such questions can be askedand answered interactively.

The job data conversion module 12403 may use job descriptive informationproduced by or passed through the ingestion facility 12402 to constructjob instance content suitable for task definition. The job dataconversion module 12403 may use the information provided by theingestion facility 12402 to query content in the library 12314 (e.g.,via the data processing facility 12030 as optionally depicted). Contentin the library that may be useful or informative of task definition mayinclude job syntax (e.g., terms that are relevant to a given job, jobtype, set of tasks and the like, such as “front end loader”,“cybersecurity”, “hi-lift jack” and others), robot types, robotcapabilities (e.g., by type, cost, availability, etc.), keyword-to-taskcross reference, workflow definition rules, job execution planformat/content/structure. Further the library may include templates forvarious task definition-related activities, such as exemplarymulti-purpose robot configurations (e.g., based on task keyword and thelike), exemplary team configurations (e.g., for performing certain typesor classes of tasks), task definitions, workflows and workflowdefinitions, exemplary job execution plan(s) and the like.

A keyword-based task lookup module 124010 may retrieve information inthe job instance storage 12408, such as task-oriented keywords and thelike and apply those to the library 12314 to potentially identifypreconfigured or templated tasks or portions thereof. As an example, ajob description may include keywords, such as “submerged” and the likethat may suggest a need for robots that can perform tasks whensubmerged. When such keywords are combined with an action “submergedexcavation”, the keyword-based task lookup facility 12410 may identifyrobot types that perform excavation and can be submerged. If adescriptor of a task in the library aligns with one or more jobdescription keywords, the task may be considered a candidate task forthe job.

In embodiments, a task definer module 12412 may process candidate tasksprovided by the task lookup module 12410 as well as information in thejob instance storage 12408 to form definitions 12304D for tasks to beperformed by one or more robots. Defining tasks may include tasks thatare predefined by standards, laws, and the like. As an example, acandidate task may include opening a manhole cover on a public way.Predefined tasks for meeting standard and/or laws and the likeassociated with such a candidate task may include notifying local lawenforcement, local public utilities, placing safety signs at specifieddistances from the open hole, marking the open hole, maintaining watchat the hole while it is opened and actively preventing unauthorizedhuman entry, and the like. Each task definition may include informationuseful for identifying a robot type for performing the task.

In embodiments, the task definition system 12304 may process task dataderived from a job request (e.g., as provided by the job request parser12302) in the context of robot types by identifying characteristics ofrobot types that align with the task data. In example embodiments, thetask definition system 12304 may determine that task data indicates acharacteristic of a robot for performing the task may include nuclearradiation tolerance (e.g., a task of inspecting a nuclear reactor core).In this example, the task definition system 12304 may generate a taskdefinition 12304D for the nuclear reactor core inspection task thatincludes at least a requirement for robot selection based on thischaracteristic. In these example embodiments, the task definition 12304Dmay further include a required degree of tolerance to nuclear radiation(number of rads, duration of exposure, and the like). The taskdefinition system 12304 may further determine that characteristics ofone or more robots (e.g., based on task information derived from the jobrequest) that may not be suitable for incorporation in a singlerobot/robot type. This determination may be based on, for example, robotcharacteristics and type data that is accessible in the library 12314.In such an example, the task definition system 12304 may define multipletasks, each with robot characteristics that are consistent with robotcharacteristic information in the library 12314. In embodiments, thetask definition system 12304 may define a task with multiple,potentially incompatible robot characteristics, optionally along with anindication of one or more portions of the task that require each type ofthe multiple incompatible robot characteristics that a fleetconfiguration system 12020 may use when configuring fleet resources,such as robots and the like. In embodiments, a task definition 12304Dmay include one or more suggestions for types of robots for performingthe task, such as based on alignment of task requirements (e.g., derivedfrom task information of a job request), robot characteristics, androbot types that may be available in the library 12314. As will beexplained below, a fleet configuration system 12020 may evaluate a taskdefinition 12304D, including any suggested robot types. Other exemplarydata that may be communicated when defining a task may include tasksequence dependencies that may be suitable for defining a workflow thatincludes the defined task. As an example, a sample preparation task maybe required to be performed after a sample taking task. Such adependency may be documented in the sample preparation task and reliedupon by the workflow definition system 12306. The task definer module12412 may save a defined task into the job instance storage where it maybe cross referenced to job descriptive data (e.g., keywords and thelike) so that future detections of the cross-referenced keywords can bequickly result in a suitable task definition.

FIG. 138 illustrates example embodiments of a fleet configuration system12020 according to some embodiments of the present disclosure. Inembodiments, a fleet configuration system 12020 provides specificsoftware, hardware, and multipurpose robot configuration requirementsfor completion of a job execution plan. An exemplary construction of afleet configuration system 12020 to provide these requirements isdepicted in the block diagram of FIG. 138 . In this exampleconstruction, a fleet configuration proxy module 12466 may beconstructed to receive task definitions 12304D from a job configurationsystem 12018. The fleet configuration proxy module 12466 may beinstantiated in association with processing of a job request by the jobconfiguration system 12018 to facilitate access to and use of fleetconfiguration system 12020 resources and systems. This and otherinstantiations of the fleet configuration proxy module are furtherdescribed in association with the job configuration system 12018 herein.The fleet configuration proxy module 12466 may process task definitionsand forward them to fleet resource identification systems, such as afleet robot operating unit identification system 12454 and a fleetnon-robot operating unit identification system 12452. Each of theseidentification systems may process the task definition data providedthrough the fleet configuration proxy, separating operational data fromfleet resource data. A task definition may describe a set of fleetresources required to perform the task, such as types of robot operatingunits (e.g., one or more special purpose robots), support resources(e.g., power systems, lighting, communication systems, and the like).The robot operating unit type identification system 12454 may providejob-specific robot operating unit demand data 12476 to the fleetconfiguration scheduler 12468. The job-specific robot operating unitdemand data 12476 may identify types and quantities of robots, specificrobot operating units (e.g., by unique identifier), robot operating unitcapabilities, and the like.

In some embodiments, a fleet configuration scheduler 12468 may respondto a job request by allocating fleet resources to meet the job requestneeds. These needs may be preprocessed, as described herein by a jobconfiguration system 12018 and specifically by the task definitionsystem 12304 to facilitate fleet configuration, allocation, andscheduling. The fleet configuration scheduler 12468 processes inputsthat describe fleet inventories, such as robot operating unitinventories 12460, and non-robot operating unit inventories 12458 toidentify candidate inventory elements for satisfying a job request.These inventories may be adjusted based on existing allocations of robotoperating units and non-robot operating units. As an example, allspecial purpose robots of a type identified in the robot operating unitjob-specific demand data 12476 may be allocated throughout a duration oftime within which a requested job is constrained to be performed. Thefleet configuration scheduler 12468 (e.g., with support from otherplatform resources such as fleet intelligence layer 12004, fleetprovisioning system 12014 and the like) may allocate, based onconditions in the job request and robot type equivalence data availableto the fleet configuration scheduler 12468, a multi-purpose robot forthe activities requested to be performed by the special purpose robot.To accomplish this allocation, a fleet intelligence layer 12004 may beprovided with information descriptive of the functionality to beprovided by the special purpose robot indicated in the job-specificdemand data 12476 and information descriptive of the tasks and/oractivities required to be performed by the special purpose robot. Othercontext, such as differences in specifications for performing tasks by aproperly configured multi-purpose robot and by the special purpose robotmay also be available to the fleet intelligence layer 12004. Through useof artificial intelligence, which may include determining an impact onan overall job request based on use of the two different robot types,the fleet intelligence 12004 may provide robot substitution guidance tothe fleet configuration scheduler 12468. This guidance may result inallocation of a multi-purpose robot and necessary configurationdata/features (e.g., end effectors and the like) for use when executinga job execution plan that corresponds to the job request that promptedthis fleet configuration scheduling activity. In an example of fleetconfiguration scheduling, a 3D printing capable robot or fleet-servicingresource (e.g., a 3D printing factory or third-party provider) may beallocated to the job to print robot parts that enable the multi-purposerobot to perform the functions of the special purpose robot (e.g., arobot arm/end effector 3D printed as a flexible/soft structure that canconform to an irregular shape for performing a task).

In embodiments, a task definition 12304D may include recommendations forone or more types of robots (e.g., based on alignment of, for example,task requirements, robot characteristics, and robot types), and apreferred type may be designated in the task definition 12304D. As anexample, a task may be suitable for performance by a multi-purpose robotor a special purpose robot (e.g., robot characteristics that align withthe task information may be found in the library 12314 for aconfiguration-specific multi-purpose robot and for a special purposerobot). While the multi-purpose robot may be suitable, a special purposerobot may be preferred due to other factors in the job request, such asan accumulated error threshold that may be exceeded by use of amulti-purpose robot, pricing, availability, and/or the like. When amulti-purpose robot type is indicated in the task definition 12304D, areference to configuration data (and/or the data itself) may also becommunicated in the task definition 12304D.

As described above, task information may be converted into a taskdefinition that may require different or at least multiple robots. As anexample, a sampling task requiring robots with different characteristicsthat is defined may be identified as SAMPLE-Tl. A first robot may beassigned by the fleet configuration system 12020 for a first portion ofthe task (e.g., SAMPLE-Tl-A for a sample site preparation activity, suchas removing objects obstructing the sample operation), and a secondrobot/robot type may be assigned for a second portion of the task (e.g.,SAMPLE-Tl-B for a sample-taking activity) and the like. When at leasttwo robot units are identified in the task, a task team designator maybe communicated. By linking a team designator to a task identifier, thefleet configuration system 12020 may consider the specific needs of theteam members to perform the task when preparing fleet resourceallocation for job execution.

The fleet configuration scheduler 12468 may rely on other fleet systems,such as a fleet provisioning module 12014 that may contribute to and/ordetermine provisioning of fleet and third-party resources and supplies.

The platform 12000 intelligence layer 12004, the fleet provisioningmodule 12014 and other fleet systems, including the fleet configurationscheduler 12468 may interact with a fleet configuration modeling system12474 that may facilitate generation of fleet configuration options12472 that can be considered by the fleet configuration scheduler 12468when configuring a fleet in response to job configuration activities andthe like. Fleet configuration modeling 12474 may provide simulation offleet configurations, such as by using fleet digital twins, which mayoptionally be associated with a digital twin system of the fleetintelligence layer 12004.

In embodiments, the fleet configuration scheduler 12468 may rely on afleet team organizer module 12470 that assists in determining/effectingteam configurations. Job-specific demand data 12476 may identify (e.g.,recommend) set(s) of robot operating units to be configured as teams.Also, job-specific demand data 12476 may indicate information that maybe indicative of configuring teams, such as co-location of robotsperforming a task and the like. The team organizer 12470 may confirmand/or designate team metadata for use when configuring a fleet. Theteam metadata may indicate team membership and time frame for themembership (e.g., from one date to another, from a start of a task untilthe task is complete, and the like).

The fleet configuration scheduler 12468 may update fleet allocation datasets (that may be used by fleet resource allocation and/or reservationcapabilities described herein), such as the fleet robot operating unitallocation data set 12462 and the fleet non-robot operating unitallocation data set 12456 with fleet configuration allocationinformation based on configuration(s) generated for the job-specificdemand data 12476 provided. The various inputs, including fleetconfiguration impacting external data 12464 (e.g., weather, locationdata, traffic data, industry standards, job-specific contextualinformation, and the like) may be processed, optionally iteratively, bythe fleet configuration scheduler 12468 to produce, among other things,fleet configurations 12478 that may be returned to an executing instanceof a job configuration system 12018 via the fleet configuration proxy12466.

FIG. 139 illustrates example embodiments of the workflow definitionsystem 12306 according to some embodiments of the present disclosure. Inembodiments, the workflow definition system 12306 may be constructed togenerate definitions of workflows for requested jobs utilizing resourcesof the fleet management platform. The construction of the workflowdefinition system 12306 may include an ingestion module 12502 thatreceives and processes task definitions 12304D that may be provided fromthe task definition system 12304 or sourced from the library 12314, andjob specific fleet configuration information 12504 that may be providedfrom job configuration system 12018 interactions with the fleetconfiguration system 12020 (e.g., via the fleet configuration proxy12305).

Ingestion of task definitions and/or fleet configuration information mayinclude aligning the fleet configuration information 12504 with one ormore task definitions 12304D. As an example of aligning tasks with fleetconfiguration information, fleet configuration information may be taggedas applying to one or more tasks in the set of task definitionsingested, such as with an identifier of the tasks or tasks. Other waysof aligning task definition(s) with fleet configuration information maybe based on timing of such ingestion so that, for example, when a fleetconfiguration reference/value is received contemporaneously with a taskdefinition the ingestion module 12502 may mark these two data items asaligned. Other ways of aligning task definition(s) with fleetconfiguration information may include one or more data values in thetask definition, which may be a data set, linked list, flat file,structured data set and the like indicating fleet configurationinformation to which the task(s) should be aligned. Fleet configurationinformation may include one more task identifiers to which the fleetconfiguration information pertains and/or should be applied whengenerating workflow definitions.

Ingestion may further include processing references (e.g., URLs,hyperlinks, external names, and the like) to workflow content in thelibrary 12314 that may be found in any of the ingested content. In anexample, a task definition may include a name of a task that is storedin the library 12314. The ingestion module 12502 may identify the nameby its syntax (e.g., a prefix may be added to a task identifier thatindicates the task is to be retrieved from the library) and/or taskdefinition structuring (e.g., a list of task names stored within asubset of the task definition that is structured to indicate the subsetof tasks are to be retrieved from the library). While the examples ofingestion herein pertain to an instance of ingestion of one or more taskdefinitions, ingestion may be performed on batches of tasks. Multipleinstances of the ingestion module 12502 may be instantiated andoperating concurrently to process a plurality of task definitions may beperformed. Optionally, a stream of tasks definitions may be received byingestion and each task in the stream is ingested in sequence.

One or more outcomes of processing by the ingestion module 12502 may bepresented to a set of workflow definition activities including a taskdependency determination module 12506 that may determine dependenciesamong tasks, such which tasks need to be performed in a sequence andwhich tasks can be performed independently of other tasks. The taskdependency determination module 12506 may also determine dependency oftasks on other factors, such as availability of fleet resources,calendar/date/time, readiness of supply materials and the like.Dependency on other factors may be identified in the task definition,such as by marking a given job state as a start point for the task. Inan example of job state task dependency, a task of processing a sampleof material may be dependent on the material being received by a samplecataloging robot and the like. Further other factor task dependency maybe attributed to a given task definition during ingestion (e.g., basedon aligning a task with a fleet configuration that sets a dependency onavailability of fleet resources, such as a special purpose robot and thelike).

A task grouping activity 12508 may process outcomes of the taskdependency activity 12506 to generate groups of tasks based on a rangecriteria, such as tasks that depend on a given task being complete(e.g., opening a building ventilation system port) may be grouped forconcurrent execution. Grouping tasks may be based on dependency on fleetresource availability, so that tasks that are dependent on a fleetresource may be grouped and performed once the resource is available.The order of performance of these grouped tasks may be based oninter-task dependency. Generally, tasks may be grouped for a range ofpurposes, such as cost savings, resource guarding, job prioritization,available job execution funds, anticipated fleet resource maintenanceneeds, earliest task start/finish time, latest task start/finish timeand the like.

A task workflow step definition activity 12510 may determine whichtask(s) can be organized into each step of one or more workflows. Basedon inter-task dependency (or lack thereof) multiple workflows may bedefined, each workflow including one or more workflow steps that aredefined in workflow step definition activity 12510. As an example ofinter-task dependency, a proscribed task, such as one driven by anelectrical safety standard, may serve as a reference point to whichother workflow development activities must conform. Referring again tothe building ventilation system inspection example referenced herein, aset of workflow steps for opening a ventilation port may be configured(with optional adaptation based on other conditions) into multipleworkflows, one for each ventilation port. Further, a workflow step, oncedefined, may be assigned to and/or referenced in a plurality ofworkflows. When dependencies exist, such as availability of a specialpurpose robot for performing a task in a workflow step, a plurality ofworkflows may themselves be made dependent. In an example, when a taskof opening a ventilation port is defined for a special purpose robot andthe job requires opening four ports, workflows that include this portopening tasks may be made dependent so that each workflow is startedonly when the required resource is available. Performance of other tasksin these workflows may be concurrent even if the initial task of openingthe port must be done sequentially due to the fleet resource utilizationdependency.

In embodiments, a defined workflow step may be an adapted variant of acandidate workflow step 12514, such as a workflow step that is retrievedfrom the library 12314. The workflow step definition activity 12510 mayrequest input from other fleet resource platform services, such as thedata processing system 12030 and/or artificial intelligence services12028 to adapt a candidate workflow step for use when defining one ormore workflow steps for a given job.

Information such as workflow step dependency may be utilized by aworkflow step linking activity 12512 that may receive step linkingrecommendation(s) 12516 from the fleet intelligence layer 12004 and thelike. Workflow step linking activity 12512 may generate a data structurethat indicates a sequence of performing defined workflow steps (e.g., aworkflow definition 12306D. The workflow definition 12306D may includedata that captures job-specific workflow information, such as workflowstep ordering, workflow step performance sequence, workflow stepindependence, step-by-step links to workflow steps, workflow successcriteria, cross-workflow dependencies, and/or the like.

In embodiments, workflow definition(s) 12306D may be stored in a jobinstance storage 12408 where they can be referenced as needed during jobconfiguration and/or job execution. They may be stored in the fleetlibrary 12314 where they can be referenced by other jobs, by thirdparties, such as job requestor and the like. They may be storedelsewhere (e.g., a cloud storage facility) based on architecturalconsiderations, such as being distributed to edge computinginfrastructure resources proximal to job deployment sites and the like.

In embodiments, workflows may be simulated as indicated in thedescription of the job configuration system 12018. Outcomes ofsimulation may be directed to, for example, the ingestion module 12502where ingestion operations, such as alignment of fleet configurationdata with task description data may be improved. Outcomes may also bepassed to as feedback 12406 to other components of the platform 12000 toimprove task definition, job configuration, fleet configuration, and/orthe like.

In a specific example, an exemplary robot fleet job may compriseinspecting a building ventilation system. The job request parsing system12302 may parse a job request and any related documents to identifyventilation system inspection routines, tasks, actions, steps,requirements, and the like. The job request parsing system 12302 mayprovide the parsed information to the task definition system 12304. Inembodiments, an inspection procedure associated with the job request mayindicate one such inspection procedure step for entering the ventilationsystem (e.g., through a wall or ceiling register and the like). The taskdefinition system 12304 may identify a plurality of tasks associatedwith the procedural step of entering the ventilation system. These tasksmay include: gathering information about the physical configuration ofthe ventilation system that may identify the location and type ofregisters available in the building, analyzing the ventilation physicalinformation to select candidate registers, determining requirements foraccessing the register (e.g., is it located behind a locked door, willentering the system through the register require lifting a robot, andthe like), tools for removing a cover/grate of the register, and thelike. Further information that may be related to one or more of thetasks for this procedural step may include, without limitation, sizelimitations of a robot entering the ventilation system (which may not bespecified in the procedure, but may require determination as a taskbased on the ventilation system entry port, based on the informationabout the physical configuration of the ventilation system, and thelike), weight limits of such one or more robots and the like. Inembodiments, tasks defined by the task definition system 12304 mayinclude data analysis tasks that may be performed by fleet resources,including resources other than individual robot operating units, such asdigital twins and the like that may operate on platform processingsystems, human fleet resources, and the like. Other routines/tasks forentering a ventilation system that may require definition may includeorienting a robot for entry. A consequence of such a determination mayresult in adding requirements for a robot to perform the task(s). Inembodiments, vertical entry may require ventilation duct grippers beingoriented at the front of the robot. A task definition 12304D may includespecifics, such as duct gripper orientation and the like that othersystems of the platform 12000, such as the fleet configuration system12020 may use when configuring aspects of a fleet. In general, adiscrete robot task definition 12304D may include (explicitly orimplicitly) a plurality of (basic/rudimentary/generic) robot movementsand/or routines optionally ordered and aggregated together to meet alow-level objective (e.g., task) of a robot fleet job. Therefore, a taskdefinition system 12304 producing task definitions 12304D for a specificrobot fleet job (e.g., inspecting a ventilation system as exemplifiedherein) may generate task definitions that embody more than genericrobot element movement, such as by aggregating and/or adapting suchrobot movements to satisfy some criteria for performing the target job,such as removing an access panel for a ventilation system. Robotoperations, such as locating and turning a fastener, gripping an accesspanel, dispositioning the removed panel, reserving the fasteners, andthe like may be generic robot routines or movements that can beaggregated and adapted into a job-specific task. These generic robotroutines or movements may be available to the task definition system12304 to facilitate defining relevant aspects of tasks based on jobrequest and related criteria. In the example of inspecting a ventilationsystem, locating a fastener on an access panel may be adapted duringoperation of this task based on details of the target access panel thatmay be identified in the task definition 12304D or may be left up to anintelligence system, such as a robot-based intelligence system and thelike for on-the-task adaptation. A basic robot action, such as turningthe fastener to remove it may be adapted based on information providedin the task definition that may define the proper end effector, torque,and length of movement. In embodiments, these adaptations may be left upto a robot control function that determines contemporaneously withperformance of the task which end effector, and the like to use.Information in the task definition 12304D may facilitate robotadjustments for gripping the access panel. This information may includean orientation of the panel, a weight of the panel, features of thepanel, size of the panel, and the like to avoid damaging the panel,while ensuring to grip it securely. A task/action of dispositioning theremoved panel may be configured with a degree of location-specificflexibility to defer to a robot operating control system that mayutilize other criteria (e.g., safety standards and practices, workplacepolicies, governance and the like) to ensure that objects in the tasklocation (e.g., furniture, windows, walls, and the like) are not damagedby the panel and pathways through the task location are not blocked ormade dangerous for humans. Such a task may therefore be interpreted bythe fleet configuration system 12020 so that a robot that includesfeatures for evaluating a deployment location, such as a vision systemand the like may be matched with the defined task. In embodiments, suchflexibility may be selected from the robot configuration library 12314.

Continuing further with the exemplary robot fleet job of inspecting abuilding ventilation system, the workflow definition system 12306 maygather information output by the task definition system 12304 and thefleet configuration system 12020 (e.g., optionally via the fleetconfiguration proxy 12305) when establishing a workflow for at least theprocedural step of entering the ventilation system. At a level ofabstraction, this procedural step may include two primary tasks: (i)removing the access panel, and (ii) entering the ventilation system.Information from the task definition system 12304 may indicate that task(i) is a prerequisite for performing task (ii). The workflow system12306 may therefore define a workflow for this portion of the requestedjob with task (i) occurring before task (ii). An additional task (iii)may include 3D image capture of the environment where entry is beingmade to the ventilation system. Information from the fleet configurationsystem 12020 about one or more robots configured for these tasks mayindicate that two robots are configured, a first robot for task (i) anda second for task (ii). The workflow system may utilize this informationto determine that an order of tasks (i), (ii), and (iii) can beoptimized by defining a workflow that has the second robot perform task(iii) while waiting for the first robot to complete task (i). If thefleet configuration information for these tasks indicated that a singlerobot is provisioned for these 3 tasks, then the workflow system maydefine an order of tasks as (iii) followed by (i) and then followed by(ii). These alternate workflow configurations responsive to informationprovided to the workflow system indicate a degree of flexibility of theworkflow system when defining workflows, such as to ensure efficient useof fleet resources and the like.

Simulation of a workflow of these three tasks via the workflowsimulations system 12308 may also provide insight into any of the taskdefinitions, fleet resource allocation, workflow definitions. As anon-limiting example, simulation of a workflow that defines an order oftasks as (i), (ii) and finally (iii) may yield that step (iii) cannot beperformed for a single robot allocation as indicated because the singlerobot operating unit performing these three tasks would be disposedinside the ventilation system at step (ii). A result of the simulationmay be provided back to at least the workflow system to rework theworkflow. In embodiments, data resulting from the simulation (e.g.,failure of performing step (iii)) may be fed back to any earlier step ina job configuration system process, such as task definition, fleetconfiguration and the like. In another example of workflow simulation,with two robots configured to perform these tasks as described above, ifthe workflow calls for 3D imaging of the task area (task (iii)) by thesecond robot contemporaneously with the first robot removing the accesspanel (task (i)), the simulation may attempt to perform a simulation ofthe 3D imaging function with, for example, a digital twin of the secondrobot. The simulation may fail if the second robot is not configured bythe fleet configuration system with the 3D imaging capability. Feedbackfrom such a simulation may result in a range of changes in jobconfiguration. Two example changes may include: (i) adjust robotconfiguration (retain the workflow and change the configuration of thesecond robot to include 3D imaging capabilities); and (ii) adjust one ormore task assignments (assign the 3D imaging function to the first robotand adjust the workflow).

In embodiments, a job execution plan 12310 for inspecting a buildingventilation system may include at least the three defined tasks (i),(ii), and (iii), fleet resource (e.g., robot configuration) andallocation information (e.g., from the fleet configuration system 12020)for each task, and a workflow defining a sequence of the three tasks.

In view of the foregoing disclosures, the fleet management platform12000 may be a stand-alone service or may be integrated into a largersystem-of-systems. Furthermore, the fleet management platform 12000 isconfigured to facilitate many different types of fleets for differenttypes of tasks. In addition to the configurations that are describedabove, some additional examples of fleets and robot operating units thatmay be configured by the fleet management platform 12000 are providedbelow.

FIG. 139 illustrates example embodiments of a multi-purpose robot 12100according to some embodiments of the present disclosure and may beapplied to the general examples of an MPR 12100 of FIG. 129 . Ingeneral, a multi-purpose robot 12100 is designed, built, configured, andoperated to maximize operational flexibility in individual and groupdeployment scenarios. In this way, a multi-purpose robot 12100 may beconfigured and reconfigured to perform certain task-specific functionsin addition to the baseline functionality of the multi-purpose robot12100. In embodiments, the MPR 12100 may be configured to operateautonomously, semi-autonomously, or using directions provided by one ormore users. In embodiments, the MPR 12100 may include a baseline system12102, a module system 12120, a robot control system 12150 and a robotsecurity system 12170. For task-specific capabilities, an MPR 12100 mayincorporate configurable and interchangeable hardware and softwaremodules provided by a physical interface module 12122 and a controlinterface module 12130 of module system 12120. These modules may mounton and interface with the control system 12150, the robot securitysystem 12170, and/or the baseline system 12102 required for robotmobility, power distribution, and the like.

In embodiments, the baseline system 12102 of an MPR 12100 includesvarious hardware, devices, interfaces, processors, software, and systemsthat perform the baseline functions of the MPR 12100. In someembodiments, the baseline system 12102 may include an energy storage andpower distribution 12104 that stores energy and delivers power to theother components of the robot, enclosures 12106 that enclose some or allof the components of the MPR 12100, an electromechanical andelectro-fluidic system 12108 that actuates and control the mechanicalcomponents of the MPR 12100, a transport system 12110 that includesmechanical components that physically move the MPR 12100 in an intendedenvironment, a vision and sensing system 12112 including a baseline setof sensors that are used in connection with performance of the baselinefunctions and/or certain task-specific functions, and a structuralsystem 12114 including one or more skeletal components configured toprovide form and structure to the MPR 12100.

As can be appreciated, the baseline system 12102 of an MPR 12100 may beconfigured in accordance with the characteristics required to operatethe MPR 12100 in certain operating environments or conditions (e.g., tooperate in heat, cold, humidity, land, sea, underwater, air, undergroundand/or the like), regardless of the tasks that the MPR 12100 may becustomized to perform. Thus, different classes of MPRs 12100 configuredfor operation in different operating environments or conditions willhave different configurations of the respective baseline system 12102 ofthe MPR 12100. For instance, an example baseline system 12102 of afour-legged terrestrial MPR 12100 designed to operate on solid ground inrainy conditions may include, for example, an IP-43 rated enclosure12106 that houses four individual mechanical legs with electric motors12112 in each leg 12110, powered by electrical energy stored in abattery and supplied by a wireless power distribution system 12104. Inanother example, an example baseline system 12102 of an aquatic robotMPR 12100 designed to operate underwater may include an IP 68-ratedenclosure 12106 houses a water-jet propulsion system that uses anelectric motor 12112 powered by electrical energy stored in a battery.In yet another example, a third baseline system 12102 of an MPR 12100designed to operate in mud may include tracked wheels 12110, where poweris supplied by a gasoline engine coupled with a hose-less hydraulicpower transmission system 12104.

In embodiments, the energy storage and power distribution system 12104of a MPR2B00 may include one or more power source(s) configured tosupply power to various components of the MPR 12100 like a hydraulicsystem, an electrical system, a nuclear system, supercapacitors,flywheels, solar cell or photovoltaic cells, fuel cells, batteries, apower cord, kinetic or piezo electric battery charging device, inductivecharging or wireless power receiver and other types of power systems. Inembodiments, the choice of the power source may depend on differentfactors like the size and shape of the MPR 12100, the environment theMPR 12100 is operating in, the tasks that the MPR 12100 needs to performand so on. In embodiments, the choice of a power source may be based onthese factors and may support wide range of use case scenarios for theMPR 12100. For example, the MPR 12100 may rely on lithium ion batterysystem while operating as a mobile robot tasked with cleaning a housebut switch to wall power supply for fixed location applications that mayconsume significant power e.g., to move heavy loads in construction orearth moving applications. In embodiments, the different components ofthe MPR 12100 may be powered by the same power source, be powered bymultiple power sources or may each connect to a different power source.

In embodiments, the power source component in the energy storage andpower distribution system 12104 includes multiple lithium-ion smartbatteries, and may include rechargeable batteries or battery packsconfigured to provide charge to other components of the MPR 12100. Theuse of smart batteries allows for a modular battery system, potentialupgrades when new chemistries become available, and monitoring of powersystem status at the individual battery level. Using multiple batteriesresults in a system that is tolerant of the failure of any singlebattery element, since such a loss only reduces the maximum availablepower and energy storage. In embodiments, the MPR 12100 may be poweredby a primary power source constituted by an AC electricity supply gridfrom a power grid and a secondary source constituted by a battery pack.In embodiments, system power is provided by a fixed source external tothe MPR 12100 using one or more power repeater coils and an integratedwireless power distribution system provides, monitors, and manages powerflow and supply to subsystems of the MPR 12100 such as sensor packages.

In embodiments, the power source in the energy storage and powerdistribution components 12104 includes a hydraulic system configured touse fluid power to drive the MPR 12100. The various components of theMPR 12100 may operate based on hydraulic fluid being stored in areservoir and transmitted through a high-pressure supply line using apump at a specified pressure and flow rate to one or more hydraulicmembers like various hydraulic motors, hydraulic cylinders, andactuators for example. The hydraulic system may transfer hydraulic powerby way of pressurized hydraulic fluid through tubes, flexible hoses, orother links between components of the MPR 12100. The particular designand components of the hydraulic system can vary and any number orcombination of valves, control systems, actuators, reservoirs, pumps orany other items can be included as desired. The typical response time ofthis type of hydraulic system is very rapid, of the order of a fewmilliseconds or less.

In embodiments, the hydraulic system is designed to utilize additivemanufacturing methods and its associated design advantages to producemanifolds and reservoirs that minimize hoses and connections that canresult in leaks and system inefficiencies. The hydraulic system mayinclude the ability for the MPR2B00 to apply repairs, service equipmentand handle emergency situations through the application work arounds. Inembodiments, the hydraulic system is designed to utilize additivemanufacturing methods and its associated design advantages to producemanifolds, reservoirs, and distribution systems that incorporate valveactuation.

In embodiments, the enclosure 12106 of an MPR 12100 may include anyhousings or other physical components that contain at least a portion ofthe MPR 12100. The structure of the enclosures 12106 may vary and maydepend on the operation that the MPR 12100 may have been designed toperform. In embodiments, the enclosure 12106 is a rectangular metal boxwith an internal space which is isolated from the environment byexternal walls having predetermined environmental resistance. Theinternal space may house various components of the MPR 12100 includingenergy storage and power distribution system 12104, electromechanicaland electro-fluidic system 12108, transport system 12110, vision andsensing system 12112, robot control system 12150, robot security system12170 and the like.

In some embodiments, the enclosure 12106 of a MPR 12100 may be designedfor robustness and ability to tolerate the external environment. Forexample, protection may be provided from water, humidity, dust,vibration, and temperature. One or more sealing mechanism may beprovided to protect against water ingress. In some instances, a waterrepellent coating may be provided. Thus, the MPR 12100 may be able totolerate external weather conditions, such as rain, wind, sun or snow.

In some embodiments, the enclosure 12106 of a MPR 12100 IP-68 compliantdenoting optimum protection against dust and water. The IP Code, orIngress Protection Code, sometimes referred to as InternationalProtection Code, IEC standard 60529 classifies and rates the degree ofprotection provided by mechanical casings and electrical enclosuresagainst intrusion, dust, accidental contact, and water. An IP rating isdenoted by two signs, that is, “IP (the first sign) (the second sign).”The first sign represents a protection rating of electric equipment andcabinets against solid foreign matters, which is represented by sevenratings from “0”, which means no protection against dust entry, to “6”,which means no dust entry inside. The second sign represents aprotection rating against water entry, which is represented by nineratings from “0”, which means no protection against water entry, to “8”,which means the optimum resistance. When no rating is determined, “X” isdenoted.

In some embodiments, the enclosure 12106 of a MPR 12100 is made of anon-conductive and heat-dissipating smart material. The material mayhelp in protecting the sensitive electronic components includingcomponents of vision and sensing system 12112 and robot control system12150.

In some embodiments, the electro-mechanical and electro-fluidic system12108 of the MPR 12100 may include a set of electrical and mechanicalcomponents configured to provide form and structure and to enableoperation of the MPR 12100. The set of electrical and mechanicalcomponents may interwork with each other to enable the MPR 12100 toperform various functions. For example, electrical components may beconfigured to provide power from power sources in the energy storage andpower distribution system 12104 to the various mechanical components.The electrical components may include various mechanisms capable ofprocessing, transferring, or providing electrical charge or electricsignals. Among possible examples, electrical components may includeelectrical wires, circuitry, or wireless communication transmitters andreceivers to enable operations of the MPR 12100. Electrical componentsmay also include electric motors including a brushed DC motor, brushlessDC motor, switched reluctance motor, universal motor, AC polyphasesquirrel-cage or wound-rotor induction motor, AC SCIM split-phasecapacitor-start motor, AC SCIM split-phase capacitor-run motor, AC SCIMsplit-phase auxiliary start winding motor, AC induction shaded-polemotor, wound-rotor synchronous motor, hysteresis motor, synchronousreluctance motor, pancake or axial rotor motor, stepper motor, or anyother type of electrical or non-electrical motor. The electric motorsmay help with moving one part relative to the other. Mechanicalcomponents represent hardware of the MPR 12100 that may enable roboticsystems to perform physical operations. The particular mechanicalcomponents may vary based on the design the MPR 12100 but may includesome basic skeletal components like a structured body connected with oneor more appendages or end-effectors through one or more joints.

In some embodiments, the MPR 12100 includes a structural system 12114constituting a plurality of joints, appendages and skeletal componentsconfigured to provide form and structure to the MPR 12100. Thestructural system 12114 may include a body, a torso, a head, legs, arms,wheels, end effectors, manipulators, gripping devices and the like. Theskeletal components of the structural system 12114 may include an innercore with male and/or female ends. The various skeletal components maybe connected to the enclosure 12106 and other skeletal componentsthrough joints, mechanical fasteners (e.g., nuts and/or bolts),actuators, hinges, latches, or other suitable mechanisms. The skeletalcomponents of structural system 12114 may provide support and allow forthe transfer of fluid, electrical power, data, or the like. The jointsmay couple together skeletal components and allow movement in one ormore degrees of freedom. The joints may allow skeletal components tomove in vertical and horizontal directions as well as rotate relative toone another. For example, the MPR 12100 may comprise one or more armmotors which may be used to move the arm with respect to the body. Inembodiments, an arm motor may comprise an actuator which may be operatedby a source of energy, typically electric current, hydraulic fluidpressure, or pneumatic pressure, and converts that energy into motion.Examples of actuators may include linear actuators, solenoids, combdrives, digital micromirror devices, electric motors, electroactivepolymers, hydraulic cylinders, piezoelectric actuators, pneumaticactuators, servomechanisms, servo motors, thermal bimorphs, screw jacks,or any other type of hydraulic, pneumatic, electric, mechanical,thermal, and magnetic type of actuator.

An MPR 12100 may be configured with zero or more legs or anothermoveable or fixed base depending on the particular application orintended use of the MPR 12100. An implementation of the MPR 12100 withzero legs may include wheels, treads, or some other form of locomotion.An implementation of the robotic system with two legs may be referred toas a biped, and an implementation with four legs may be referred as aquadruped. Other implementations with six or eight legs may also bepossible. The structure of the MPR 12100 including the enclosure 12106,body, shape, size, skeletal components and material etc. may vary andmay depend on the operation that the MPR 12100 may have been designed toperform. For example, when developed to carry heavy loads, the MPR 12100may have a wide body that enables placement of the load. Similarly, whenconfigured to reach high speeds, the MPR 12100 may have a narrow, smallbody made of light weight material.

In some embodiments, an MPR 12100 may be structured to mimic the humanbody, such that the MPR 12100 includes a torso, a head, two arms, andtwo legs. The actuators may work like muscles and joints and may allowthe skeletal components to rotate relative to one another in a mannersimilar to the bones in a human body rotating about a joint. Forexample, the joints may be configured to move skeletal components in amanner similar to the movement of hands, fingers, elbows, waists, knees,wrists, shoulders, and/or the like. The build material may includebiologically inspired artificial skin equipped with sensors to detectcontact, acceleration, proximity and temperature.

In embodiments, the transport system 12110 of a MPR 12100 may includeone or more body motors which may be used to move the MPR 12100 throughone or more transportation conveyances. The transportation conveyancesmay be configured to facilitate the movement of the MPR 12100 across asurface. In some embodiments, a transportation conveyance may comprise awheel, a caster, a tread or track, a low friction pad or bumper, a lowfriction plate, a ski, a pontoon, or any other suitable deviceconfigured to reduce the friction between the MPR 12100 and the surfaceover which it is desired to be moved. In further embodiments, atransportation conveyance may comprise a propeller, miniaturized jetengine, or any other air transportation enabling device which may allowthe MPR 12100 to fly or function similar to a drone air craft. Infurther embodiments a transportation conveyance may comprise a fin, awater jet, a screw, or any other water transportation enabling devicewhich may allow the MPR 12100 to move on or below the surface of water.In further embodiments a transportation conveyance may comprise arocket, and ion drive, a gyroscope, or any other space transportationenabling device which may allow the MPR 12100 to move in space.

In embodiments, the vision and sensing system 12112 may include a rangeof sensors in the MPR 12100 acting as input mechanisms to collectinformation from the environment. This sensing information is providedto the robot control system 12150 which processes such information toactuate other subsystems including the energy storage and powerdistribution system 12104, the electromechanical and electro-fluidicsystem 12108, the transport system 12110 and the structural system12114. The vision and sensing system 12112 thereby enables the MPR 12100to monitor and navigate its environment including interacting with andmanipulating one or more objects in its environment. Examples of avision and sensing system 12112 are described in detail in conjunctionwith FIG. 142 .

The robot control system 12150 includes various hardware, devices,interfaces, processors, software, and systems for controlling theoperation and behavior of the MPR 12100. For example, the control system12150 may cause the MPR 12100 to move to a specific location byfollowing a path and avoiding obstacles in the path. As another example,the control system 12150 may cause the MPR 12100 to collaborate withothers or interact with its environment including grasping ormanipulating one or more objects in its environment.

The robot control system 12150 may read from the sensors to update theactuators which act as output mechanisms to drive the joints, the arms,the legs, the end-effectors and the like. The robot control system 12150provides precise motion control of the MPR 12100, including control overthe fine and gross movements needed for manipulating an object. Thecontrol system 12150 is able to independently control each robotic jointand other skeletal components of the structural system 12114 inisolation from the other joints and skeletal components, as well as tointerdependently control a number of the joints to fully coordinate theactions of the multiple joints in performing a relatively complex worktask.

The robot control system 12150 may communicate with other systems of theMBR, other robots, and/or the fleet management platform 100 via wired orwireless connections, and may further be configured to communicate withone or more users. For example, the control system 12150 may receive aninput (e.g., from a user or from another robot) indicating aninstruction to navigate to a location. The control system 12150 may thusserve as an interface between different components of the MPR 12100,such as between sensors and actuators, between mechanical and electricalcomponents, as well as between the MPR 12100 and a user.

In embodiments, the robot control system 12150 includes and/or mayleverage intelligence layer 12140, performance management system 12146,task management system 12144, data processing system 12142, modulemanagement system 12148, communications system 12152, navigation system12154, safety and compliance system 12156, motion planning system (MPS)12158, and/or controller 12160. It is appreciated that the foregoingdescription of the robot control system 12150 is applicable to othertypes of robots as well, including special purpose robots and/orexoskeleton robots.

In embodiments, the intelligence layer 12140 provides a framework forproviding intelligence services and help enable the MPR 12100 to makedecisions, predictions, classifications, or the like. In embodiments,the intelligence layer 12140 receives requests from the robot controlsystem 12150, or the baseline system 12102 of the MPR 12100, and/or thelike to provide a specific intelligence (e.g., a decision, aclassification, a prediction or the like). For example, the intelligencelayer may be tasked with making a decision on controlling the motion ofthe MPR 12100 based on environment data (e.g., maps, coordinates ofknown obstacles, images, and/or the like). In embodiments, the frameworkprovided by the intelligence layer 12140 may be configured as part of abroader intelligence layer extending to fleet 4D00 and/or platformlevels, as described elsewhere in the disclosure.

In embodiments, the intelligence layer 12140 may include an intelligencelayer controller 12141 and an artificial intelligence (AI) service12143. In embodiments, the intelligence layer controller 12141 may beconfigured to determine the type of services to be provided byartificial intelligence services 12143 and, in response, may determine aset of governance standards and/or analyses to be applied by theartificial intelligence services 12143. The intelligence layer 12140 ofthe MPR 12100 (or SPRs or exoskeletons) may include some or all of theintelligence services 12143 of the intelligence system described above.Furthermore, in some embodiments, the robot-level intelligence layer12140 may be configured to escalate an intelligence request to a higherlevel (e.g., the fleet level, edge device, or the fleet managementplatform 12000) when the MPR 12100 cannot perform the task autonomously.Example embodiments of a robot-level intelligence layer 12140 along withits components and subsystems are described in detail in conjunctionwith FIG. 140 .

In embodiments, the performance management system 12146 is configured tomanage the performance of one or more robotic resources includinghealth, energy, thermal flows, network and the like. In embodiments, theperformance management system 12146 may include a thermal managementservice 12161, an energy management service 12162, a monitoring andnotifications service 12163, a network management service 12164 and/or apredictive maintenance service 12165.

In embodiments, the thermal management service 12161 may use robotsensors, task historical data, ambient conditions, materialcharacteristics, form factors, and/or the like and a set of acceptableoutcomes to drive optimization algorithms that manage thermal flows in amulti-purpose robot 12100. This could be used to actively manage thermalconditions or optimize heat transfer to maintain acceptable operatingconditions. In embodiments, the thermal management service 12161 mayhelp reclaim waste heat energy. For example, waste heat could be movedto actively cool hotter components, used with emerging nanoscale orother thermoelectric devices, etc. In embodiments, thermal managementservice 12161 may leverage robot sensor data, task historical data,ambient conditions, material characteristics, form factors, etc. plus aset of acceptable outcomes to drive optimization algorithms (e.g.,quantum optimization algorithms and/or neural network optimizationalgorithms) that design and manage operation of heat transfer componentslike fins, vanes, biomimicking elements, meshes, fabrics, fans, etc. inthe MPR 12100.

In embodiments, the energy management service 12162 helps a robotintelligently manage available energy resources and maintain systemcapability while working in dynamic operating environments. For example,upon discovering that grid energy may not be available and the robotneeds to conserve the available battery, an energy management service ofthe MPR 12100 may activate one or more energy storing and recoveringdevices like flywheels, capacitors, supercapacitors, hydro-pneumaticaccumulators and the like. The devices enable the MPR 12100 to harvestthe energy during the braking phase of a motor—which energy is usuallywasted—store it, and provide it back to the system when necessary. Inembodiments, energy sharing devices may share the braking energy of amotor for driving other (non-braking) motors or actuators on a commonnetwork. In embodiments, the energy management service 12162 may includemachine learning-based predictive energy management that automaticallyactivates energy harvesting and sharing devices and deactivatesnon-essential functions on need basis.

In embodiments, the monitoring and notification service 12163 may beconfigured to monitor for and report on one or more conditions of theMBR 12100. In some of these embodiments, the monitoring and notificationservice 12163 performs summary calculations on tracking metrics ofvarious resources to discover out-of-routine characteristics. In someexample embodiments, monitoring and notification service 12163 mayperform vibration analyses that are indicative of robot health includingconditions of one or more motors or mechanical components. In some ofthese embodiments, the monitoring and notification service 12163 mayleverage machine-learned models that are trained to diagnose certainconditions of a robot (e.g., failing components, loose components,and/or the like) to predict the existence or likely occurrence of thecertain conditions. In embodiments, the monitoring and notificationservice 12163 may leverage one or more machine learned models includingvision models for monitoring, discovering and predicting emergingrobotic fault modes. In embodiments, monitoring and notification service12163 may also provide alerts and notifications upon discovering anyout-of-routine characteristics to a user. For example, upon predictingthat the battery is about to get completely depleted, a monitoring andnotification service 12163 may provide alerts and notifications to theuser using a voice message. Additionally or alternatively, themonitoring and notification service 12163 may use email, text message,instant message, phone call, and/or other communication (e.g., using theInternet or other data or messaging network) to transmit thenotification to a computing device of the user (e.g., a computer, tabletcomputer, smart phone, telephone, mobile phone, PDA, TV, gaming consoleand the like). In embodiments, the error notifications may provideoptions for the user stopping operations or making adjustments to one ormore settings associated with the error notification. In embodiments, amonitoring and notifications service 12163 may provide a user withcustom reports including analytics based on real-time and historicaldata about statuses and/or diagnoses of various of the MPR's 12100resources.

In embodiments, the network management service 12164 includes a set ofpolicies, procedures, workflows, and responsibilities assigned toimprove or maintain optimal network performance. In embodiments, thenetwork management service 12164 may assess network flow data, packetdata and network infrastructure metrics to identify and mitigateinstances of bottlenecks or network issues that may affect the operationof the MPR 12100.

In embodiments, the predictive maintenance service 12165 may predictwhen one or more components or subsystems of the MPR 12100 shouldreceive maintenance based on simulation data derived from digital twinsystem or real-world data derived from monitoring and notification12163. In embodiments, the predictive maintenance service 12165 mayaccess the intelligence layer 12140 of the MPR 12100 to predict theanticipated wear and failure of components of the MPR 12100 by reviewinghistorical and current operational data, thereby reducing the risk ofunplanned downtime and the need for scheduled maintenance. For example,in embodiments the predictive maintenance service 12165 may provide anintelligence request to the intelligence layer that includes currentoperational data obtained from the MPR 12100 (e.g., sensor data,environmental data, and/or the like), whereby the intelligence layer12140 (e.g., the machine-learning service) may leverage one or moremachine-learning models (e.g., prediction models, classification models,neural networks, and/or the like) to identify a potential failure of acomponent of the MPR 12100. In embodiments, the machine learning modelsmay be trained using data about robot specifications, parameters,maintenance outcomes, environmental data, sensor data, run information,notes to perform failure forecasting and predictive maintenance.Additionally or alternatively, the machine learning services may includea clustering algorithm to identify the failure pattern hidden in thefailure data to train a model for detecting uncharacteristic oranomalous behavior. The failure data across multiple robots and theirhistorical records may be clustered to understand how different patternscorrelate to certain wear-down behavior and develop a maintenance planresonant with the failure.

In another example, the predictive maintenance service 12165 mayleverage a digital twin service of the intelligence layer 12140 tosimulate operation of the MPR 12100 in a digital twin (e.g., in theenvironment that the MPR 12100 is operating in or will be operating in),whereby the digital twin simulation may uncover potential wear and tearof the MPR2B00 and/or a potential failure of components of the MPR12100. In these examples, over-servicing or over-maintaining the MPR12100 may be mitigated, thereby reducing costly downtime, repairs orreplacement of the MPR 12100 or its components, by addressing suchissues in a proactive or just-in-time manner.

In embodiments, the task management system 12144 coordinates between jobexecution system of the fleet operations system 12002, library 12314,vision and sensing system 12112 and the intelligence layer 12140 toexecute a task. Task management system 12144 is described in greaterdetail throughout the disclosure.

In embodiments, the data processing system 12142 may include dataprocessing resources that may be centralized and/or distributed and mayinclude general purpose chipsets, specialized chipsets, and/orconfigurable chipsets. Data processing system 12142 may include one ormore processors providing scalable computation capabilities for robotcontrol system 12150 including various intelligence resources in theintelligence layer 12140. The processors in the data processing system12142 may communicate with a number of peripheral devices via a bussystem. The peripheral devices may include a data stores including forexample, a memory subsystem for storage of instructions and data and afile storage subsystem providing persistent storage for program and datafiles, a network interface system providing an interface to outsidenetworks, a data management system with capabilities including dataallocation, data caching, data pruning and data management and access toand control of intelligence and data resources and user interface inputand output devices.

In embodiments, the data processing system 12142 includes a datahandling service 12166 and a data processing service 12167. The datahandling service 12166 is configured to store, retrieve, and otherwisemanage the data of the MPR 12100. In embodiments, the data handlingservice 12166 accesses a set of data stores 12168 and/or libraries12169, whereby the data handling service 12166 writes and reads datafrom the data stores 12168 and/or libraries 12169 on behalf of othercomponents of the MPR 12100. In embodiments, the data processing service12167 performs data processing operations on behalf of variouscomponents of the MPR 12100. For example, the data processing service12167 may perform database operations (e.g., table joins, retrieves,etc.), data fusion operations, and the like.

In embodiments, the module management system 12148 coordinates the useand configuration of various control interface modules 12130 andphysical interface modules 12122 as described below.

In embodiments, the communication system 12152 is constructed to enableefficient, high speed electronic and wireless communication amongcomponents and subsystems of the MPR 12100 as well as communication ofthe MPR 12100 with fleet operation system and its elements as describedherein, external data sources 12036, third party systems (e.g., via anInternet and the like), robot operating units, support systems andequipment, human fleet resources and the like. The communication system12152 may include or provide access to one or more network types, suchas wired, wireless and the like that may support various data protocols,such as Internet Protocol (IP), Bluetooth communication protocol,wireless communication protocols (e.g., IEEE 802, 4G communicationprotocol, 5G communication protocol), and/or the like. In embodiments,the communication system 12152 may leverage intelligence services toconfigures, prioritizes and controls data and resources to varioussystems internal and external to the MPR 12100.

In embodiments, the navigation system 12154 allows the MPR 12100 tonavigate known, partially known and unknown environments by establishingits own position and orientation within the environment (localization)while creating a map of the environment (mapping) as it moves around inthe environment. In some embodiments, the navigation system 12154 mayemploy Simultaneous Localization and Mapping (SLAM) for autonomousnavigation of robots by recognizing its own position using a sensorwhile mapping the environment. The SLAM algorithm creates a map of thesurrounding environment at the initial position and estimates theposition of the robot and the map of the surrounding environment byrepeating the process of finding the position of the moved robot basedon the created map. The navigation system 12154 may utilize additionalor alternative navigation algorithms as well.

In embodiments, the navigation system 12154 may work with vision andsensing system 12112 to generate one or more images of the MPR 12100within its environment. Such images may be clicked by cameras and imagesensors of the vision and sensing system 12112 and may include one ormore images clicked using the camera 12608 with the conformable variablefocus liquid lens 12612. The images may be to the machine vision system12618 may utilize one or more neural network models including CNN orRCNN to locate the MPR 12100. Additionally, multiple other sensors likemotion sensor, depth sensor, proximity sensor, LIDAR etc. may be used inconjunction with one another to localize the MPR 12100 more accuratelywithin its environment.

Further, the in some embodiments, the navigation system 12154 mayincrementally build and/or update a map of the environment where the“map” denotes a field of static objects that surround the robot. The MPR12100 traverses through this map and attempts to measure range to eachobject, either through imaging, laser range finding, or ultrasonics, andcontinuously updates both the location of the detected objects and itsown location, with respect to the objects.

In embodiments, the navigation system 12154 may also work with themotion planning system 12158 to plan the path of the robot and/or thetask management system 12144 (in conjunction with the robot-levelintelligence layer 12140) to determine an optimal navigation policywithin the environment. In some embodiments, the navigation system 12154coordinates with robot control system 12150 to generate controlinstructions to effectuate movement of one or more actuators or motorsin accordance with the navigation policy enabling the MPR 12100 navigateits environment.

In embodiments, the safety and compliance system 12156 is configured toperform safety assessments, including mechanical safety, electricalsafety and functional safety. In embodiments, the safety and compliancesystem 12156 is configured to ensure compliance with one or more safetystandards and generate workflow and process control documentation toobtain certificates of conformance from one or more standards orcertifications authorities. In embodiments, safety and compliance system12156 ensures compliances with one or more Standards Authorities includeInternational Organization for Standardization (ISO), UnderwritersLaboratories (UL), TUV SUD, ANSI (American National Standards Institute)and the like. For example, ISO 10218 describes four separate robot-humancollaborative operating modes to ensure that humans are not exposed tounacceptable risks. Similarly, ISO/TS 15066 provides technicalspecification and engineering guidance for users to conduct riskassessments when installing collaborative robot. In some embodiments,the safety and compliance system 12156 may leverage the intelligenceservices in making safety assessments.

In embodiments, the motion planning system 12158 may be configured tocontrol the motion of the MPR 12100 or portions thereof and build anoptimal collision free path for the MPR2B00. Example embodiments of themotion planning system 12158 are described in further detail inconjunction with FIG. 140 .

In embodiments, the controller 12160 in the control system may drive theactuators in the transport system 12110, end effectors, or other anyother electro-mechanical component of the MPR2B00, thereby enabling theMPR 12100 to perform at least a portion of a task. In embodiments, thecontroller 12160 may receive signals from one or more of the navigationsystem 12154, the task management system 12144, the motion planningsystem 12158, the communication system 12152, and/or the modulemanagement system 12148 to determine a control signal to issue to animplicated actuator, which the controller 12160 may output to theimplicated actuator.

In embodiments, the module system 12120 may be configured to provide oneor more task specific capabilities to the MPR 12100 using one or moreconfigurable and interchangeable hardware and software modules. Inembodiments, the module system 12120 includes the control interfacemodule 12130 and/or a physical interface module 12122. In embodiments,the control interface module 12130 may include one or more softwaremodules to provide connectivity, power, security, sensing, computing andartificial intelligence (AI) like capabilities. In embodiments, thephysical interface module 12122 may include one or more end effectors,or end of arm tooling systems configured to provide the MPR 12100 withthe ability to perform certain operational tasks.

In embodiments, a control interface module 12130 includes one or moreinterfaces that are configured to receive respective modules configuredto enhance various capabilities of the MPR2B00 such as sensingcapabilities, power capabilities, networking capabilities, edgecomputing capabilities, and/or the like. Such capabilities may enablethe MPR2B00 to perform specialized functions such as specialized sensingand evaluation and to work in environments with edge and networkingconstrains, power constraints, mobility constraints and the like.

In embodiments, the control interface module 12130 may includenetworking modules 12131, sensor modules 12132, computing modules 12133,security modules 12134, AI modules 12135, communications modules 12136and user interface modules 12138. In embodiments, the control interfacemodule 12130 receives one or more sensor modules 12132. The sensormodules that are used to configure the MPR 12100 may depend on the tasksand jobs that the MPR 12100 is being configured to perform. Forinstance, the sensor modules 12132 may include weight sensors,environment sensors (e.g., temperature, humidity, ambient light, motionsensors, vision sensors (e.g., cameras, lidar sensors, radar sensors,etc.), or other suitable sensors. In embodiments, the sensor modules12130 may be specialized chips, such as a lab-on-a-chip package, anorgan-on-chip package, or the like.

In embodiments, the control interface module 12130 incorporates one ormore modular, removable and replaceable lab-on-a-chip sensor package toprovide chemical and biological sensing. The lab-on-a-chip sensorpackage may enable the MPR2B00 to perform chemical and diagnostictesting including chemical assays, microbiological culture assays,immunoassays and nucleic acid assays and may be useful for environmentalconditions testing, water and gas particle analysis, first respondertesting, toxicology, military, disaster, and related applications.

In embodiments, the control interface module 12130 incorporates one ormore modular, removable and replaceable organ-on-a-chip sensor packagetailored to sense and evaluate biological and related hazards. Theorgan-on-a-chip sensor package may be a microfluidic culture device thatsimulates the architecture, mechanics, functions and physiologicalresponse of living human organs, including the lung, intestine, kidney,skin, bone marrow and blood-brain barrier, among others. Some exampleuse-cases include first-responders, operator health, pandemic, andrelated applications.

In embodiments, the control interface module 12130 incorporates one ormore modular, resettable and replaceable collision sensors packageconfigured to detect potential collisions and disengage or send a signalto the robot to stop or reverse movement when a collision is detected.The collision sensor package may help with preventing, reducing oreliminating damage to the end effector, tooling and the parts orproducts being processed.

In embodiments, the control interface module 12130 incorporates one ormore modular, removable and replaceable AI-on-a-chip package configuredfor a specific task or policy, and integrated to work with a variety ofvisual and other sensor inputs. Some examples of task specificAI-on-a-chip packages include machine vision packages, natural languageprocessing packages, image classification packages, video analysispackages, predictive analysis packages, optimization packages, controlpackages or packages configured for implementing one or more policies inpolicy libraries. In embodiments, the modular AI-on-a-chip packages maybe configured for training of one or more of machine learning models,reinforcement learning models, neural networks, policy networks and thelike. In embodiments, the modular AI-on-a-chip packages may beconfigured for specific environments like warehouses, manufacturingenvironments, agricultural and farming environments, shipping andlogistics environment, medical environments and the like. The modularAI-on-a-chip packages may be trained with domain-specific models thatare built for the specific environment or use cases. For example, thepackage may include a natural language processing model specificallycustomized for understanding language used in an agricultural orwarehouse environment. As another example, the model may be trained on aset of medical images and used for identifying microbial infections. Inembodiments, the control interface module 12130 incorporates one or moremodular, removable and replaceable AI-on-a-chip package configured forspecific environments including environments with low or intermittentpower, extreme environmental conditions, high temperature and low heatdispersion, and the like. In embodiments, the modular AI-on-a-chippackages may be configured to autonomously optimize local resourcesbased on a task specific requirement including optimization for compute;storage; network; energy; heating/cooling capacity; battery capacity;human resources capacity; space; additive manufacturing capacity and thelike.

In embodiments, the modular AI-on-a-chip packages may be trained withmodels to execute and govern robotic process automation, such asrecognizing situations (bottlenecks in warehouse, congestion/lines instore, thin/sparse customer mix in part of an environment), classifyingand recognizing objects/faces/products/emotions, setting demand-sideparameters (price, promotion, advertising location); managingsupply-side interactions including governing onboard chatbotinteractions, managing recommendation engine for recommending a basketof complementary products and the like. In embodiments, the modularAI-on-a-chip packages may be trained with models to analyzephysiological, neurological, emotional, cognitive state of a user andtailor the response of the MPR 12100 based on such state. For example,the package may analyze facial expressions, speech, tone, body movementsof a user to determine the state, analyze the state information toderive information on customer interest, response, preference etc. andthen feed such information to edge devices for content delivery, productrecommendations, advertising, and the like. In embodiments, the modularAI-on-a-chip packages may be trained with models to analyze securitythreat vectors and other vulnerabilities to the MPR 12100 or the roboticfleet. For example, the package may use biometric analysis, behavioralmodeling, facial and voice recognition, for enabling authentication;learning models for recognizing and preventing attacks by malware,spyware, ransomware, viruses, worms, trojans and the like;classification, clustering or regression models for threat intelligence,anomaly detection, network and end-point security etc. In embodiments,the modular AI-on-a-chip packages may be trained with models to analyzeweather conditions, light, temperature, water usage or soil conditionscollected from farms in agricultural planning by determining seed andcrop choices and optimizing utilization of farming resources includingland, water and nutrition. The MPR 12100 may for example, use theinformation to follow a planting and nutrition routine, performphenotyping for selective breeding provide optimized wavelengths oflight for crops using AI-controlled LED lights. In embodiments, themodular AI-on-a-chip packages may be trained with models to detectdiseases, pests, weed, nutritional deficiencies in soil or crops onagricultural farms. For example, the MPR2B00 may utilize propeller orminiaturized jet engine of transport system to fly over the farm,capture images of the farm using cameras of the vision and sensingsystem and then use the modular AI-on-a-chip package to identify problemareas and potential improvements. For example, the images may show thepresence of unwanted plants or weeds. The MPR 12100 may then makedecisions about treatment with herbicides or may select one or moreend-effectors for eliminating the weeds. In embodiments, the modularAI-on-a-chip packages may be trained with models to monitor and harvestcrops, plants, fruits and vegetables of various shapes and sizes. Forexample, the package may utilize machine vision and other sensors foridentifying the crops ready to be harvested. The package may alsoinclude trained policies for navigating the farm, estimating theposition and orientation of crops relative to the MPR 12100, graspingfruits and vegetables of different shapes and sizes, select suitable endeffectors for selective harvesting, and finally storing or packaging theharvested fruits and vegetables. In embodiments, the modularAI-on-a-chip packages may be trained with models to manage a controlledclosed loop environment for an aquaponics system based on needs ofplants and fish. For example, an example module AI-on-a-chip package mayreceive sensed oxygen levels in an aquatic environment and may determinewhether the water is sufficiently oxygenated, under-oxygenated, orover-oxygenated. In embodiments, the modular AI-on-a-chip packages maybe trained with models for optimizing 3D printing parameters.

In embodiments, the control interface module 12130 may receive multiplemodular, removable and replaceable combinations of modules to performcertain tasks. For example, in some embodiments, the control interfacemodule 12130 may receive a lab-on-a-chip capability to detect gases andAI-on-a-chip capability for machine vision. The MPR 12100 may forexample, use such a package for gas leak detection and isolation inover-ground and underground gas pipelines. In this example, the MPR12100 may travel along the pipeline and analyze gas concentrations inclose proximity to potential leak points. Upon determining a gas leak,the MPR 12100 may use cameras and IR sensors to click images, machinevision capability to locate the leak and policy libraries to identifyone or more policies to fix the leak.

In embodiments, the physical module interfaces 12122 receive (orotherwise connect to) auxiliary physical modules that alter the physicalactions that may be taken by MPR 12100 and/or the physical operation ofthe MPR 12100. Some examples of physical module interfaces 12122 includeend effectors 12124, motive adapters 12126, 3D printer adapters 12128and the like. End effectors 12124 includes devices or tools that may beconnected to the end of the arm of MPR 12100 for manipulating objects oraccomplishing one or more tasks. For example, different end effectorsmay be used for gripping and grasping, lifting and placing, palletizing,brushing, drilling, inspecting, and/or testing objects. The MPR 12100may be configured with one or more of the end effectors, such that theone or more end effectors may be selected based on multiple factorsincluding the task(s) to be performed; the size, shape, surface andweight of the object to be manipulated; environment of the objectincluding the material clearance available around the object; availablepower supply; the precision or accuracy required in the task; and thelike. It is appreciated that the end effectors that are used by the MPR12100 may be selected by the fleet management platform 12000 duringconfiguration and/or by the MPR 12100 while deployed.

In some example embodiments, end effectors may include grippers forgripping and grasping objects for wide range of material handlingapplications right from stacking large boxes to handling tiny, delicateelectronic components. In some example embodiments, fingers or jaws maybe attached to grippers to grip or hold the object as well as pick upand place objects, for example on an assembly line, conveyor system orother automated system. For example, parallel grippers may have twofingers disposed parallel to each other that may close on an object tohold and grip the same, angled grippers may have fingers at a variety ofvariety of different angle openings like three fingers offset by 120°,suction grippers may have one or more suction cups for engaging asurface of an object and using a negative or suction pressure or vacuumto grasp the object; electro-magnetic grippers may be used for grippingmetal objects, hydraulic grippers powered by hydraulic fluids may beused for heavy duty applications like lifting heavy objects, softgrippers may mimic human fingers to pick and manipulate delicate objectsof differing shapes and sizes like fresh fruits and vegetables,Bernoulli grippers may use airflow to adhere to an object withoutphysical contact and may be used for handling sterile material toprevent contamination and so on. In embodiments, the grippers mayinclude sensors aiding the gripper in locating, handling, andpositioning products. In embodiments, the grippers may includeaccessories like force torque sensors and compliant force feedbacksystems for force-controlled processes requiring application of preciseforce. In embodiments, the grippers may be powered by compressed air,vacuum or electricity. In some example embodiments, the end effectors12124 may have a wide variety of process tooling devices attached forvarious applications including arc welding, spot welding, paintspraying, machining, drilling, water-jet cutting, flaming, riveting,grinding, deburring, assembling, additive manufacturing, injectionmolding and/or the like.

In embodiments, motive adapters 12126 may include suitable modularcomponents that allow the MPR 12100 to traverse certain environmentsand/or conditions. For example, motive adapters 12126 may includedifferent wheel sets, movable legs, fins, jets, turbines, or othersuitable means of transport.

In embodiments, 3D printer adapters 12128 incorporate an integrated setof additive manufacturing capabilities for printing on a need basis. Forexample, the additive manufacturing capabilities may include printingtools, such as agricultural tools or parts, constructions tools orparts, packaging tools or parts, replacement parts, and/or othersuitable additive manufacturing capabilities that allow a robot to printitems on a need basis. In these embodiments, the additive manufacturingcapabilities may include suitably dimensioned printing devices forprinting items, as well as any materials needed for the printing.

The foregoing descriptions of different modules are provided for exampleof respective types of physical modules and control modules. It isunderstood the physical modules interfaces 12122 and control moduleinterfaces 12130 may receive other additional or alternative moduleswithout departing from the scope of the disclosure.

FIG. 140 is an example architecture of the robot control system 12150depicting detailed view of various components thereof, according to someembodiments of the present disclosure. In embodiments, the intelligencelayer 12140 receives requests from a set of intelligence layer clientsand responds to such request by providing intelligence services to suchclients (e.g., a decision, a classification, a prediction or the like).At the robot level, such clients may include various components andsubsystems of the robot control system 12150 including the performancemanagement system 12146, the task management system 12144, the modulemanagement system 12148, the navigation system 12154, the motionplanning system 12158, and the like; various components of the baselinesystem 12102 including the energy storage and power distribution 12104,the electromechanical and electro-fluidic system 12108 the transportsystem 12110, the vision and sensing system 12112 and the structuralsystem 12114 or other suitable systems of MPR 12100 including the modulesystem 12120 or the robot security system 12170.

As an example, the intelligence layer 12140 may take as input sensordata including environment data, video camera streams, maps, audiostreams, images, coordinates of known obstacles, and/or the like fromvision and sensing system 12112. The intelligence layer 12140 may thencoordinate with motion planning system 12158 to make one or moredecisions about the motion of MPR 12100 or portions thereof, coordinatewith the navigation system 12154 to make decisions about navigating inthe environment and coordinate with task management system 12144 to makedecisions about performing one or more tasks. The controller 12160 inthe robot control system 12150 may then generate the controlinstructions to drive the actuators enabling the MPR 12100 to move,navigate in the environment and perform various tasks.

In embodiments, the motion planning system (MPS) 12158 may be configuredto control the motion of MPR 12100 or portions thereof (e.g., endeffectors, end of arm tools). In embodiments, a motion planning system(MPS) 12158 may specify a series of transition that the MPR 12100 canfollow getting from a “start state” and navigating to a “goal state”without colliding with any obstacles in the environment. In embodiments,the start state and the goal state may be determined based on the taskor sub-task to be performed. The start state and goal state may beexpressed as positions of the robot, poses of the robot, geolocations ofthe robot, and/or the like.

In some embodiments, the MPS 12158 may take as input one or more imagesand other sensor data from a vision and sensing system as well asinformation indicative of the “start state” and the “goal state” (e.g.,from the navigation system 12154 or other suitable component). Inembodiments, the MPS 12158 may then build a motion plan for the robot.In some embodiments, the motion plan is a motion planning graph thatrepresents the geometric structure of the environment with the states ofthe MPR 12100 as nodes and transitions between the states as edges ofthe graph). In embodiments, a graph search may be performed to find apath between the nodes representing the “start state” and the “goalstate”. The MPS 12158 may also perform collision assessment determiningthe probability of collision between the MPR 12100 and one or moreobstacles in the path and assign cost values to edges of the graph basedon the probability of collision for the corresponding transition. TheMPS 12158 may perform a least cost analysis on the motion planning graphto determine a set of transitions or path from the “start state” to the“goal state”. In embodiments, the MPS 12158 may coordinate withintelligence layer 12140 and navigation system 12154 to implement anavigation policy with the identified set of transitions or path. TheMPS 12158 may also coordinate with controller 12160 to generate controlinstructions to actuate one or more actuators or motors in the MPR 12100so as to execute the motion plan.

In embodiments, the MPS 12158 may be configured to identify an optimalcollision free path in a 3D workspace while taking into account variouskinematic, geometric, physical and temporal constraints as well asaccount for additional constraints including complex tasks (e.g.,manipulation of objects) and uncertainty (the movement of the one ormore obstacles). Collision detection determines if the volume in 3Dspace swept by the MPR 12100 moving from one state to another collideswith any obstacles. The surface of the swept volume and the obstaclesmay be represented as polygons and collision detection involvescomputing whether these polygons intersect.

In embodiments, the MPS 12158 may utilize one or more machine learningmodels 12664 in the intelligence layer 12140 to adapt the motion plan toreal time changes in the environment. For example, the motion plan maybe adapted based on the changes in task performed by the MPR 12100,change in end effectors 12124 and the like. In embodiments, the MPS12158 may improve its motion planning efficiency by using transferlearning to leverage learning from one task to a related task.

In embodiments, the MPS 12158 may receive sensor data from one or moresensors of the vision and sensing system 12112 to determine any movingobstacles and may leverage one or more machine learning models 12664 topredict the trajectory of the moving obstacle in the environment basedon the machine learning models 12664. The MPS 12158 utilizes thepredicted trajectory information to compute the cost function whileconsidering the probability and cost of collision with the movingobstacle.

In embodiments, the MPS 12158 may utilize a 3D path planning algorithmfor determining the optimal path. For example, sampling-based algorithmsmay determine feasible paths for the robot's motion using informationfrom a graph that consists of randomly sampled nodes and connected edgesin the given configuration space. Such randomized approaches have astrong advantage in terms of quickly providing solutions to complexproblems, such as in a high-dimensional configuration space. Examples of3D path planning algorithms that may be used by the MPS system includevisibility graph, random-exploring algorithms such as rapidly exploringrandom tree, Probabilistic Road Map, optimal search algorithms (such asDijkstra's algorithm, A* algorithm) and bioinspired planning algorithms.

In embodiments, the navigation system 12154 utilizes a path (e.g., anoptimal path) determined by MPS 12158, along with a pre-trainednavigation policy from the task management system 12144 to build anavigation strategy for the MPR 12100. In some embodiments, thenavigation system 12154 coordinates with the robot control system 12150to generate control instructions to effectuate movement of one or moreactuators or motors in accordance with the navigation strategy enablingthe MPR 12100 navigate its environment. The navigation actions of theMPR 12100 may be evaluated by the reinforcement learning system 12668 inan iterative manner to constantly update the navigation policy.

In embodiments, the task management system 12144 coordinates between thejob execution system 12022 of the fleet operations system 12002, library12314, vision and sensing system 12112, and one or more services of therobot-level intelligence layer 12140 to execute a task. In some exampleembodiments, the task management system 12144 may refer to policylibraries to identify one or more pre-trained policies that may beapplied for completing a task upon receiving a task request (e.g., froma user, from the fleet management platform, and/or from another robot).For example, upon receiving a request for moving an object from oneplace to the other, the task management system 12144 may identifygrasping policy and navigation policy to complete the task. The taskmanagement system 12144 may also work with the vision and sensing system12112 to analyze visual and sensor information and past operatinghistory to evaluate one or more objects that may be used in the task anddetermine one or more operations necessary to perform the assigned taskon that object. For the example task of moving an object, the problem ofgrasping an object may be more complex when there is no past operatinghistory or policy and the MPR 12100 is encountering the object for thefirst time (e.g., not encountered during training). Moreover, the righttechnique to grasp may differ based on the object characteristics. Forexample, the points at which to grasp the object and the force that maybe applied while grasping may be very different for different objects(e.g., depending on consistency, fragility, shape, size, and/or thelike). The MPR 12100 may need to work with a very wide variety ofobjects with different shapes or forms like glasses, boxes, boxes withside handles, markers, flowerpots, manufacturing parts, machine tools,desks, chairs, lamps and the like and may require different techniquesand accessories to grasp and pick up such objects. The task managementsystem 12144 may leverage the intelligence layer 12140 to identifyobject characteristics and adapt the policy based on suchcharacteristics. For example, the force applied during grasping anobject may be adjusted based on whether the object is made of delicatematerials like glass or ceramic as opposed to when the object is made ofmetal. As another example, the MPR 12100 may use side handles to grasp abox when such handles are available. Accordingly, the task managementsystem 12144 may also work with the module management system 12148 toidentify and select a suitable end effector 12124 or other accessoryrequired to complete the task.

In embodiments, when a suitable end-effector is not found, the taskmanagement system 12144 may leverage the intelligence layer 12140 (e.g.,machine learning services, RPA services, and/or the like) to determineand/or design an end effector 12124 or other accessory for executing thetask, which may be subsequently ordered or printed. In the latterscenario, the task management system 12144 can utilize an additivemanufacturing system and its associated design advantages, to print thesuitable end-effector that meets the task requirements andspecifications as defined by the task management system 12144.

In embodiments, the task management system 12144 may include one or morepolicy libraries that define a set of pre-trained policies forperforming common robotic tasks. The policies are simply the sequence ofactions that need to be taken by the MPR 12100 for performing a task.Some examples for common tasks for which policies may be providedinclude, navigating, grasping, lifting, transporting, counting, sorting,stacking, cleaning, twisting, bending, compacting, drilling, polishing,loading/unloading, assembling/disassembling, packaging/unpackaging,palletizing/depalletizing, grinding, welding, painting, sealing,planting, harvesting, cutting, pruning, weeding, and/or the like. Inembodiments, the policy libraries may include multiple additive ornested learning loops for complex or multi-step tasks. For example,transporting the object from source to destination may involve graspingand lifting the object, and then navigating to the destination andplacing the object there. In embodiments, the policy libraries may referto task definitions available in library 12314 to ensure consistencywith the overall job assignment.

The policies may be defined and updated in any suitable manner. In someembodiments, the policies may be defined by a human user (e.g., aprogrammer). In some embodiments, the task management system 12144 maywork with the intelligence layer 12140 (e.g., an RPA service) to learnand optimize policies based on the quality of task completion (wherequality may be measured by metrics such as breakage, task completionrate, safety, accuracy, etc.). In some embodiments, the policies may bepre-trained using training data collected from expert demonstrations.For instance, the training data for welding may be obtained from anexpert welding professional engaged in the act of welding. The data maybe obtaining from real-world setting like a manufacturing workshop orfrom a controlled environment. In some embodiments, the policies may bepre-trained using training data collected from simulation environments.For instance, the training data for grasping may be obtained using adigital twin system performing simulations using the arm and one or moreof end effectors.

In embodiments, the policies may be pre-trained on a wide variety ofobjects and may be adapted based on characteristics of the object onwhich the policy is applied. For example, to train the grasping policy,the digital twin system 12630 may perform simulations on differentobjects including glasses, boxes, boxes with side handles, markers,flowerpots, manufacturing parts, machine tools, desks, chairs. Also,transfer learning may be used for adapting or tuning data collected forone task on another related task. For example, transfer learning mayreuse a model developed for one task as the starting point for a modelon a second related task.

In some embodiments, the intelligence layer 12140 may employ transferlearning for domain adaptation. For example, one or more transferlearning algorithms may be used for adapting the data collected by thedigital twin system in the simulation environment to the real-worldenvironment. In embodiments, the intelligence layer 12140 may employadversarial training for domain adaptation. For example, a GenerativeAdversarial Networks (GAN) may be used to generate synthetic data forthe real-world environment, which is then used for training. Also,specialized neural networks like Domain-Adversarial Neural Network(DANN) by Ganin et al may be used for domain adaptation.

Robot-Level Intelligence Layer

In embodiments, the robot-level intelligence layer 12140 of the MPR12100 may be configured as part of a broader intelligence system (e.g.,the intelligence services system 12200 of FIG. 130 ) as described above.In embodiments, the robot-level intelligence layer 12140 providesintelligence services to the MPR 12100, thereby enabling the MPR 12100to make decisions, predictions, classifications, or the like. Inembodiments, the robot-level intelligence layer 12140 may includecapabilities to perform some or all of the intelligence services thatare consumed by the MPR 12100 and/or may be configured to requestintelligence services from an external source (e.g., another robot, anedge device, and/or the fleet management platform).

In embodiments, the intelligence layer 12140 may include theintelligence layer controller 12141 and a set of artificial intelligence(AI) services 12143. In embodiments, the intelligence layer controller12141 may include an analysis management module 12600, a set of analysismodules 12610, and a governance library 12620. In embodiments, theanalysis management module 12600 receives a request for an artificialintelligence service and determines the governance standards and/oranalyses implicated by the request. In embodiments, the analysismanagement module 12600 may determine the governance standards thatapply to the request based on the type of decision that was requestedand/or whether certain analyses are to be performed with respect to therequested decision. For example, a request for a control decision thatresults in the MPR 12100 navigating to a nuclear waste treatment sitemay implicate a certain set of governance standards that apply, such assafety standards, legal standards, quality standards, regulatorystandards, financial standards or the like, and/or may implicate one ormore analyses regarding the control decision, such as a risk analysis, asafety analysis, an engineering analysis, or the like. In embodiments,the governance standards may be defined as a set of standards librariesstored in a governance library 12620. In embodiments, the governancelibrary 12620 may define conditions, thresholds, rules, recommendations,or other suitable parameters by which a decision may be analyzed. Insome embodiments, the analysis management module 12600 may determine oneor more analyses that are to be performed with respect to a particulardecision and may provide corresponding analysis modules 12610 thatperform those analyses to the artificial intelligence service 12143. Inembodiments, the analysis modules 12610 may include modules that areconfigured to perform specific analyses with respect to certain types ofdecisions, whereby the respective modules are executed by the dataprocessing system 12142 that hosts the instance of the intelligencelayer 12140. Continuing the example of the decision for the MPR 12100navigating to a nuclear waste treatment site, the level of risk andhazard at the site may need to be analyzed to make the navigationdecision. Non-limiting examples of analysis modules 12610 may includerisk analysis module(s), security analysis module(s), decision treeanalysis module(s), ethics analysis module(s), failure mode and effects(FMEA) analysis module(s), hazard analysis module(s), quality analysismodule(s), safety analysis module(s), legal analysis module(s),financial analysis module(s) and/or other suitable analysis modules.

Artificial intelligence services 12143 may include a digital twin system12630, a machine vision system 12618, a machine-learning (ML) system12632, a robotic process automation (RPA) system 12652, a naturallanguage processing (NLP) system 12656, an analytics system 12660,and/or a neural network system 12662. The machine learning system 12632may further include machine learning models 12664 and reinforcementlearning system 12668.

The digital twin system 12630 may be constructed to generate digitaltwins for MPR 12100, robotic subsystems like the electromechanical andelectro-fluidic system 12108, the transport system 12110, the vision andsensing system 12112 etc., robotic components like batteries, sensors,valves, actuators, motors, end effectors etc., robotic policies likenavigating, grasping, lifting, transporting etc. The digital twins ofthe MPR 12100 may have a visual user interface, e.g., in the form of 3Dmodels, and/or may consist of system specifications or ontologiesdescribing the architecture, including components and their interfacesof the MPR 12100. The digital twin may be configured to simulateoperation of the MPR 12100 so as to continuously capture the keyoperational metrics and may be used to monitor and optimize theperformance of the MPR 12100 in real time. The robot digital twin mayalso be configured to communicate with one or more users, twins or otherrobots via multiple communication channels such as speech, text,gestures, and the like. For example, the digital twin may receivequeries from a user about the MPR 12100, generate responses for thequeries and communicate such responses. Further, the digital twin system12630 may be configured with interfaces, such as APIs and the like forreceiving information from the operating environment of the MPR 12100.

In embodiments, the digital twin system 12630 may be used to simulatethe behavior of the MPR 12100 or one or more of its components orsubsystems. For example, the behavior of the MPR 12100 while grasping aglass bottle and moving it from source to destination may be predictedand optimized by the intelligence layer 12140. The insights gained fromanalysis and simulation using digital twins may be passed onto areinforcement learning agent for improvement of these processes.

In embodiments, multiple digital twins of the components and subsystemsof the MPR 12100 may be integrated thereby aggregating data across thevalue chain network to generate a digital twin for the MPR 12100 and todrive not only entity-level insights but also system-level insights.Similarly, the digital twins of policies may combine to form a digitaltwin of a multi-step task or a job twin. For example, the digital twinfor transporting may be seen as comprised of digital twins of grasping,lifting and navigating.

The machine vision system 12618 includes software to enable the MPR12100 extract information from digital images to recognize one or moreobjects in the environment of the MPR 12100. The machine vision system12618 may execute one or more machine learning algorithms to perform oneor more machine vision tasks including object classification, objectdetection, scene classification, pose detection, semantic segmentation,instance segmentation and image captioning and so on. The machine visionsystem may include pre-trained machine learning models to execute thedifferent machine vision tasks including a neural network like,convolutional neural network (CNN), transformer network, Region-basedCNN, fast RCNN, mask RCNN and the like.

Machine Learning System

The machine learning system 12632 may define one or more machinelearning models 12664 for performing analytics, simulation, decisionmaking, and predictive analytics related to data processing, dataanalysis, simulation creation, and simulation analysis of one or morecomponents or subsystems of the MPR 12100. In embodiments, the machinelearning models 12664 are algorithms and/or statistical models thatperform specific tasks without using explicit instructions, relyinginstead on patterns and inference. The machine learning models 12664build one or more mathematical models based on training data to makepredictions and/or decisions without being explicitly programmed toperform the specific tasks. In example implementations, machine learningmodels 12664 may perform classification, prediction, regression,clustering, anomaly detection, recommendation generation, and/or othertasks.

In embodiments, the machine learning models 12664 may perform varioustypes of classification based on the input data. Classification is apredictive modeling problem where a class label is predicted for a givenexample of input data. For example, machine learning models can performbinary classification, multi-class or multi-label classification. Inembodiments, the machine-learning model may output “confidence scores”that are indicative of a respective confidence associated withclassification of the input into the respective class. In embodiments,the confidence scores can be compared to one or more thresholds torender a discrete categorical prediction. In embodiments, only a certainnumber of classes (e.g., one) with the relatively largest confidencescores can be selected to render a discrete categorical prediction.

In embodiments, machine learning models 12664 may output a probabilisticclassification. For example, machine learning models may predict, givena sample input, a probability distribution over a set of classes. Thus,rather than outputting only the most likely class to which the sampleinput should belong, machine learning models can output, for each class,a probability that the sample input belongs to such class. Inembodiments, the probability distribution over all possible classes cansum to one. In embodiments, a Softmax function, or other type offunction or layer can be used to turn a set of real values respectivelyassociated with the possible classes to a set of real values in therange (0, 1) that sum to one. In embodiments, the probabilities providedby the probability distribution can be compared to one or morethresholds to render a discrete categorical prediction. In embodiments,only a certain number of classes (e.g., one) with the relatively largestpredicted probability can be selected to render a discrete categoricalprediction.

In embodiments, machine learning models 12664 can perform regression toprovide output data in the form of a continuous numeric value. Asexamples, machine learning models can perform linear regression,polynomial regression, or nonlinear regression. As described, inembodiments, a Softmax function or other function or layer can be usedto squash a set of real values respectively associated with a two ormore possible classes to a set of real values in the range (0, 1) thatsum to one. For example, machine learning models 12664 can performlinear regression, polynomial regression, or nonlinear regression. Asexamples, machine learning models 12664 can perform simple regression ormultiple regression. As described above, in some implementations, aSoftmax function or other function or layer can be used to squash a setof real values respectively associated with a two or more possibleclasses to a set of real values in the range (0, 1) that sum to one.

In embodiments, machine learning models 12664 may perform various typesof clustering. For example, machine learning models may identify one ormore previously-defined clusters to which the input data most likelycorresponds. In some implementations in which machine learning modelsperforms clustering, machine learning models can be trained usingunsupervised learning techniques.

In embodiments, machine learning models 12664 may perform anomalydetection or outlier detection. For example, machine learning models canidentify input data that does not conform to an expected pattern orother characteristic (e.g., as previously observed from previous inputdata). As examples, the anomaly detection can be used for frauddetection or system failure detection.

In some implementations, machine learning models 12664 can provideoutput data in the form of one or more recommendations. For example,machine learning models 12664 can be included in a recommendation systemor engine. As an example, given input data that describes previousoutcomes for certain entities (e.g., a score, ranking, or ratingindicative of an amount of success or enjoyment), machine learningmodels can output a suggestion or recommendation of one or moreadditional entities that, based on the previous outcomes, are expectedto have a desired outcome

As described above, machine learning models 12664 can be or include oneor more of various different types of machine-learned models. Examplesof such different types of machine-learned models are provided below forillustration. One or more of the example models described below can beused (e.g., combined) to provide the output data in response to theinput data. Additional models beyond the example models provided belowcan be used as well.

In some implementations, machine learning models 12664 can be or includeone or more classifier models such as, for example, linearclassification models; quadratic classification models; etc. Machinelearning models 12664 may be or include one or more regression modelssuch as, for example, simple linear regression models; multiple linearregression models; logistic regression models; stepwise regressionmodels; multivariate adaptive regression splines; locally estimatedscatterplot smoothing models; etc.

In some examples, machine learning models 12664 can be or include one ormore decision tree-based models such as, for example, classificationand/or regression trees; chi-squared automatic interaction detectiondecision trees; decision stumps; conditional decision trees; etc.

Machine learning models 12664 may be or include one or more kernelmachines. In some implementations, machine learning models 12664 can beor include one or more support vector machines. Machine learning models12664 may be or include one or more instance-based learning models suchas, for example, learning vector quantization models; self-organizingmap models; locally weighted learning models; etc. In someimplementations, machine learning models can be or include one or morenearest neighbor models such as, for example, k-nearest neighborclassifications models; k-nearest neighbors regression models; etc.Machine learning models 12664 can be or include one or more Bayesianmodels such as, for example, naïve Bayes models; Gaussian naïve Bayesmodels; multinomial naïve Bayes models; averaged one-dependenceestimators; Bayesian networks; Bayesian belief networks; hidden Markovmodels; etc.

In some implementations, machine learning models 12664 can be or mayinclude one or more artificial neural networks (also referred to simplyas neural networks). A neural network can include a group of connectednodes, which also can be referred to as neurons or perceptrons. A neuralnetwork can be organized into one or more layers. Neural networks thatinclude multiple layers can be referred to as “deep” networks. A deepnetwork can include an input layer, an output layer, and one or morehidden layers positioned between the input layer and the output layer.The nodes of the neural network can be connected or non-fully connected.

Machine learning models 12664 can be or include one or more feed forwardneural networks. In feed forward networks, the connections between nodesdo not form a cycle. For example, each connection can connect a nodefrom an earlier layer to a node from a later layer.

In some instances, machine learning models 12664 can be or include oneor more recurrent neural networks. In some instances, at least some ofthe nodes of a recurrent neural network can form a cycle. Recurrentneural networks can be especially useful for processing input data thatis sequential in nature. In particular, in some instances, a recurrentneural network can pass or retain information from a previous portion ofthe input data sequence to a subsequent portion of the input datasequence through the use of recurrent or directed cyclical nodeconnections.

In some examples, sequential input data can include time-series data(e.g., sensor data versus time or imagery captured at different times).For example, a recurrent neural network can analyze sensor data versustime to detect or predict a swipe direction, to perform handwritingrecognition, etc. Sequential input data may include words in a sentence(e.g., for natural language processing, speech detection or processing,etc.); notes in a musical composition; sequential actions taken by auser (e.g., to detect or predict sequential application usage);sequential object states; etc.

Example recurrent neural networks include long short-term (LS™)recurrent neural networks; gated recurrent units; bi-direction recurrentneural networks; continuous time recurrent neural networks; neuralhistory compressors; echo state networks; Elman networks; Jordannetworks; recursive neural networks; Hopfield networks; fully recurrentnetworks; sequence-to-sequence configurations; etc.

In some examples, machine learning models 12664 can be or include one ormore non-recurrent sequence-to-sequence models based on self-attention,such as Transformer networks. Details of an exemplary transformernetwork can be found athttp://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf.

In some implementations, machine learning models 12664 can be or includeone or more convolutional neural networks. In some instances, aconvolutional neural network can include one or more convolutionallayers that perform convolutions over input data using learned filters.

Filters can also be referred to as kernels. Convolutional neuralnetworks can be especially useful for vision problems such as when theinput data includes imagery such as still images or video. However,convolutional neural networks can also be applied for natural languageprocessing.

In some examples, machine learning models 12664 can be or include one ormore generative networks such as, for example, generative adversarialnetworks. Generative networks can be used to generate new data such asnew images or other content.

Machine learning models 12664 may be or include an autoencoder. In someinstances, the aim of an autoencoder is to learn a representation (e.g.,a lower-dimensional encoding) for a set of data, typically for thepurpose of dimensionality reduction. For example, in some instances, anautoencoder can seek to encode the input data and the provide outputdata that reconstructs the input data from the encoding. Recently, theautoencoder concept has become more widely used for learning generativemodels of data. In some instances, the autoencoder can includeadditional losses beyond reconstructing the input data.

Machine learning models 12664 may be or include one or more other formsof artificial neural networks such as, for example, deep Boltzmannmachines; deep belief networks; stacked autoencoders; etc. Any of theneural networks described herein can be combined (e.g., stacked) to formmore complex networks.

Machine learning models 12664 may include one or more clustering modelssuch as, for example, k-means clustering models; k-medians clusteringmodels; expectation maximization models; hierarchical clustering models;etc.

In some implementations, machine learning models 12664 can perform oneor more dimensionality reduction techniques such as, for example,principal component analysis; kernel principal component analysis;graph-based kernel principal component analysis; principal componentregression; partial least squares regression; Sammon mapping;multidimensional scaling; projection pursuit; linear discriminantanalysis; mixture discriminant analysis; quadratic discriminantanalysis; generalized discriminant analysis; flexible discriminantanalysis; autoencoding; etc.

In some implementations, machine learning models can perform or besubjected to one or more reinforcement learning techniques such asMarkov decision processes; dynamic programming; Q functions orQ-learning; value function approaches; deep Q-networks; differentiableneural computers; asynchronous advantage actor-critics; deterministicpolicy gradient; etc.

Reinforcement Learning is a machine learning technique for learningoptimal behavior in an environment by taking actions and gettingfeedback, similar to how humans and animals learn by interacting withtheir environments. The typical reinforcement learning approach includesan agent (say robot control system 12150) that observes its environment,evaluates its current state (e.g., robot velocity, distance to an objectin front), and selects an action (e.g., provide control instruction toactuator or motor, adjust velocity, change direction and the like). Uponcarrying out an action, the agent is presented with, in addition to itsnew state, a reward (e.g., +10 for allowing sufficient space between therobot and an obstacle in front of it and −10 for allowing insufficientspace) which provides some indication of the success of the action. Thegoal for a reinforcement learning agent is to learn an optimal policy orbehavior that maximizes the expected cumulative reward.

Reinforcement learning system 12668 includes one or more reinforcementlearning algorithms for evaluating various states, actions and rewardsin determining optimal policy for executing one or more tasks by the MPR12100.

RPA system 12652 enables the MPR 12100 automate workflows as well as anyrepetitive tasks and processes. In embodiments, the RPA system 12652 maymonitor human interaction with various systems to learn patterns andprocesses performed by humans in performance of respective tasks. Inembodiments, an RPA system 12652 may learn to perform certain tasksbased on the learned patterns and processes, such that the tasks may beperformed by the RPA system 12652 in lieu or in support of a humandecision maker.

NLP system 12656 provides the MPR 12100 with the ability to parse one ormore conversational voice instructions provided by a human user toperform one or more tasks as well as communicate with the human user. Inembodiments, the NLP system 12656 may be configured as part of, mayleverage or may be included in NLP system 4D24 described in conjunctionwith FIG. 4 . The NLP system 12656 may leverage one or more neuralnetworks from the neural network system 12662 including feed forwardneural networks, convolutional neural networks (CNN), recurrent neuralnetworks (RNN), long short-term memory (LSTM), transformer neuralnetworks and the like for performing various natural language processingfunctions. Example implementations of an NLP system 12656 are describedin greater detail elsewhere in the disclosure (e.g., with respect toFIG. 104 and related description).

In embodiments, the artificial intelligence services 12143 may includeand/or provide access to an analytics system 12660. In embodiments, ananalytics system 12660 is configured to perform various analyticalprocesses on data output from the MPR 12100 or one or more components orsubsystems. For example, the analytics system 12660 may perform dataanalytics on thermal and vibration data generated by the MPR2B00 over aperiod of time for anomaly, detection, system failure detection,predictive maintenance and for avoiding costly downtime and disruptionof operation of the MPR2B00. In another example, the analytics system12660 may analyze sensor data of the MPR 12100 to generate insightsabout things like general health of the MPR 12100 efficiency of one ormore tasks performed by the MPR 12100, optimal positions and setting forthe MPR 12100 and so on.

Neural Networks (or Artificial Neural Networks) are a family ofstatistical learning models inspired by biological neural networks andare used to estimate or approximate functions that may depend on a largenumber of inputs and are generally unknown. Neural networks represent asystem of interconnected “neurons” which send messages to each other.The connections have numeric weights that can be tuned based onexperience, making neural nets adaptive to inputs and capable oflearning.

Neural network system 12662 include one or more neural networksincluding feed forward neural networks, convolutional neural networks(CNN), recurrent neural networks (RNN), long short-term memory (LSTM)neural networks, gated recurrent unit (GRU) neural networks,self-organizing map (SOM) neural networks (e.g., Kohonen self-organizingneural networks), Autoencoder (AE) neural networks, Encoder-Decoderneural networks, modular neural networks, or variations, hybrids orcombinations of the foregoing, or combinations with reinforcementlearning (RL) systems or other expert systems, such as rule-basedsystems, model-based systems (including ones based on physical models,statistical models, flow-based models, biological models, biomimeticmodels, and the like). Examples of neural networks and neural networksystems 12662 have been described in more details elsewhere in thedisclosure (e.g., FIGS. 93-107 ).

FIG. 141 schematically depicts an example architecture of the robotcontrol system 12150 that utilizes data from multiple sensors in thevision and sensing system 12112 to learn about the environment toimplement policies and drive control for one or more components of thebaseline system 12102 including energy storage and power distributionsystem 12104, the electromechanical and electro-fluidic system 12108, orthe transport system 12110 to perform a task.

In embodiments, the MPR 12100 may acquire sensor data from one or moresensors 12602 and extract “state information” about the position of theMPR 12100 with respect to the environment 12604 and one or more objects12606. For example, the MPR 12100 may use camera 12608 to capture imagesof objects 12606. An additional vision sensor may be mounted at aposition different from that of the camera 12608 to capture image datafrom multiple viewpoints. The camera and vision sensors may generateimages related to shape, color, depth, and/or other features ofobject(s) that are in the line of sight of the sensors. The image datamay be processed and the machine vision system 12618 may execute one ormore machine learning algorithms including the CNN variants describedabove for object detection. Data from additional sensors (e.g., tactilesensor, sound sensor and/or gas sensor) may be combined to help build amore accurate model of the world in order for the MPR 12100 to navigateand behave more successfully in its environment. In embodiments, Kalmanfilters and data fusion techniques may be used for combining the datafrom multiple sensors.

In embodiments, the intelligence layer 12140 may coordinate with policylibraries in the task management system 12144 and controller 12160 togenerate the control instructions for performing one or more tasksincluding navigation, object grasping, sorting cleaning,loading/unloading, packaging/unpackaging, assembly,palletizing/depalletizing and the like.

Upon the control system 12150 receiving an input (e.g., from a user orfrom another robot) indicating one or more tasks to be performed, theintelligence layer 12140 may select one or more policies from the policylibraries in the task management system 12144 to implement. For example,upon receiving an instruction to grasp an object placed in theenvironment of the MPR 12100, the intelligence layer 12140 may determinethat the MPR 12100 needs to use navigation policy for navigating to thelocation of the object followed by grasping policy to grasp the object.The intelligence layer 12140 may use the sensor data from one or moresensors 12602 to determine “state information” describing informationextracted from a scene in the environment of the MPR 12100. The stateinformation may include images or image streams from one or more visionsensors, information collected from other sensors like gas sensor,tactile sensor and sound sensor. The state information may also includeinformation obtained after analysis of sensor information and may forexample, include presence of one or more objects in the environment,name and type of the objects, the distance and position of the objectson a map including a target object to be grasped with respect to the MPR12100, the material properties of the target object and the like.

In embodiments, the intelligence layer 12140 may then take one or moreactions based on one or more policies in response to the stateinformation. For example, the intelligence layer 12140 may determinethat the environment includes two objects and the MPR 12100 needs tomove 100 meters to reach the target object while avoiding an obstacleobject located at a distance of 10 meters. The navigation policy mayprovide navigational actions and guide the MPR 12100 to reach the targetobject while avoiding collision with the obstacle object. The graspingpolicy may then guide the MPR 12100 about action steps to grasp thetarget object. In embodiments, the policy libraries may use machinelearning including reinforcement learning to define the differentpolicies for performing the various tasks.

Based on the output of policy libraries in the task management system12144, the robot control system 12150 may then develop and providecontrol instructions for one or more actuators or control devicesassociated with the MPR 12100 to implement the policies and drive one ormore components of the electromechanical system 12108, the transportsystem 12110 or the energy storage and power distribution system 12104.For example, the control instructions may effectuate movement of one ormore motors of the transport system 12110 to navigate to a location inthe environment in accordance with the navigation policy. As anotherexample, the control instructions may effectuate movement in one or moreactuators in arm joints or end effectors to grasp the target object inaccordance with the grasping policy.

The term actuator encompasses a mechanical or electrical device thatcreates motion, in addition to any driver(s) that may be associated withthe actuator and that translate received control instructions into oneor more signals for driving the actuator. Accordingly, providing acontrol instruction to an actuator may comprise providing the controlinstruction to a driver that translates the control instruction intoappropriate signals for driving an electrical or mechanical device tocreate desired motion. The MPR 12100 may have multiple degrees offreedom and each actuator or motor may control actuation within one ormore of the degrees of freedom responsive to the control instructions.

FIG. 142 illustrates an example vision and sensing system 12112according to some embodiments of the present disclosure. The vision andsensing system 12112 include a range of sensors 12602 configured toreceive information from the environment 12604 of the multi-purposerobot 12100 and enable the MPR 12100 to interact with one or moreobjects 12606 in its environment. For example, vision sensors maycapture image data within a field of view which may assist the MPR 12100with environment recognition and navigation. Some examples of sensorsmay include one or more cameras, LIDARs, RADARs, SONARs, thermalimaging, hyperspectral imaging, illuminance sensors, force sensors,torque sensors, velocity sensors, acceleration sensors, positionsensors, proximity sensors, gyro sensors, sound sensors, motion sensors,location sensors, load sensors, temperature sensors, touch sensors,depth sensors, ultrasonic range sensors, infrared sensors, chemicalsensors, magnetic sensors, inertial sensors, gas sensors, humiditysensors, pressure sensors, viscosity sensors, flow sensors, objectsensors, tactile sensors and the like. In embodiments, sensors may bemounted on directly on non-actuable components of the robot like thehead or on actuable components like the arms or the end-effectors. Inembodiments, sensors may be physically separated from the MPR 12100 orlocated within the environment 12604 in which the MPR 12100 isoperating.

In embodiments, the vision and sensing system 12112 may monitor theenvironment 12604 in real time, and detect obstacles, elements of theterrain, weather conditions, temperature, or other aspects of theenvironment. The various sensors 12602 are configured to work in a widerange of environmental conditions and may capture data related of one ormore objects 12606 in the environment 12604, such as size, shape,profile, structure, speed, distance, or orientation of the objects12606. Some examples of sensors 12602 that may work to capture differentdata in various environments include monographic cameras (e.g., forcapturing image data), stereoscopic cameras (e.g., for 3D vision), RADAR(e.g., for long-range object detection, distance determination, or speeddetermination), LIDAR (e.g., for short-range object detection, distancedetermination, or speed determination), SONAR (e.g., for underwaterobject detection, distance determination, or speed determination),ultrasonic sensors (e.g., for bright light and very dark environmentsand to sense glass or other transparent surfaces), GPS (e.g., forposition information), IMU (e.g., for orientation information), and thelike.

In embodiments, the vision and sensing system 12112 may then coordinatewith the robot control system 12150 to process the captured sensing dataand make a sequence of decisions or devise a policy about actions to beperformed by the MPR 12100. The decisions may for example, relate toactivation or deactivation of one or more components of theelectromechanical and electro-fluidic system 12108, movement of the MPR12100 by the transport system 12110, distribution of power to certaincomponents of the MPR 12100 by energy storage and power distributionsystem 12104 and the like.

Referring now to FIG. 142 , a camera 12608 is configured to captureimages of objects 12606 located within a field of view of the camera12608. The camera 12608 may be a standard digital camera (i.e., camerasincluding CCD or CMOS sensors), stereoscopic camera, infrared imagesensor, time of flight (TOF) camera, structured light camera, and thelike having an electrical power/control connection and an opticalelement like a lens 12612. The lens 12612 may be a conformable variablefocus liquid lens configured to adjust various optical parametersincluding lens shape, focal length, liquid materials, specularity,color, environment, lens arrangement via for example, control signalsreceived via the electrical power/control connections. In embodiments,the control connections may include electrical, hydraulic, pneumatic,mechanical, thermal or magnetic controls. The conformable liquid lens12612 may include an auto-focus capability helping it to quickly adjustits focal length and enabling recognizing objects in dynamicenvironments like when the object 12606 or the MPR 12100 are moving;recognizing three dimensional (3D) objects by capturing depth data;recognizing tiny objects; recognizing objects in a power constrained ornetwork constrained environment; and so on.

The raw image data captured by the camera 12608 that may be in variousforms including RGB images, thermal images, point clouds is thentransmitted to pre-processor 12614 to perform data pre-processingincluding data transformations, filtering, de-noising, aggregation,artifact reduction, compression, analog to digital conversion,preliminary feature recognition and so on. The image data is then sentto an image processing engine 12616 for further processing for example,identifying objects 12606 in the images as well as determining theirlocation or orientation. The image processing engine 12616 may interfacewith a machine vision system 12618 within the intelligence layer 12140of robot control system 12150. The machine vision system 12618 mayexecute one or more machine learning algorithms to perform one or moremachine vision tasks including object classification, object detection,scene classification, pose detection, semantic segmentation, instancesegmentation and image captioning and so on. The machine vision systemmay include pre-trained machine learning models to execute the differentmachine vision tasks. In embodiments, machine vision system 12618 mayemploy one or more neural network-based models for processing of imagedata.

In embodiments, the vision and sensing system 12112 includes a dynamicvision system having artificial intelligence for learning on a trainingset of outcomes, parameters, and data collected from the conformablevariable focus liquid lens 12612 to recognize an object. In embodiments,the dynamic vision system is controlled by and/or optimized with inputfrom the artificial intelligence in the intelligence layer 12140, suchas wherein artificial intelligence learns on a set of machine visionoutcomes to adjust the dynamic vision system to capture visualinformation in a manner that improves outcomes, such as recognitionoutcomes, prediction outcomes, and the like.

In embodiments, the vision and sensing system 12112 includes a dynamicvision system that comprises an optical assembly with conformablevariable focus liquid lens 12616; the robot control system 12150configured to adjust one or more optical parameters and data collectedfrom the optical assembly in real time; and the data processing system12142 that dynamically learns on a training set of outcomes, parametersand data collected from the optical assembly to train a set of machinelearning models 12664 to control the optical assembly to optimize thecollection of data for processing by the set of machine learning models.In embodiments, a first model is used to optimize collection of signalsby the optical assembly and a second model is used to operate on thesignals to achieve a desired machine vision outcome. In embodiments, theoutcome is a recognition outcome, a classification outcome, or aprediction outcome.

The dynamic vision capabilities provided by the vision and sensingsystem 12112 may enable the MPR 12100 in identifying and manipulating atarget object for use in robotic assembly lines where object depth,orientation, position and motion may be inferred for improved objectidentification. The dynamic vision capabilities may also enable the MPR12100 in simultaneous localization and mapping, which is a technique forestimating the position of the robot with respect to its surroundingswhile mapping the environment at the same time.

In embodiments, the vision output from the vision and sensing system12112 may be temporally combined with output from other sensors in theMPR 12100 using conditional probabilities to create a combined view ofthe target object that is richer and includes information about theposition, orientation and motion of the object in the environment.

In embodiments, the dynamic vision capability of the vision and sensingsystem 12112 may integrate into or with a set of value chain network(VCN) entities for quality control inspections and sorting objects in aproduction assembly line or logistics chain wherein the conformableliquid lens 12612 is configured to quickly adjust focus to accommodatefor, recognize and sort objects located at various working distances orobjects of different heights.

Referring to FIGS. 104-142 , according to some example implementations,a fleet management platform having wireless power routing and managementfor robot instrumentation and related electronics may also facilitateconfiguring and operating robots with modular, removable organ-on-chipsensor robot sub-assemblies. In embodiments, power for organ-on-chipsub-assemblies may be delivered and managed wirelessly to meet a widerange of robot deployments, including mobile environments where primarypower for the robot is provided by a replaceable battery pack and powerfor the organ-on-chip is optionally provided by a sub-assembly-specificbattery pack. In embodiments, power sharing and routing of power amongthe battery packs may be performed and managed wirelessly, such as by arobot-local power management facility. The platform may facilitateperforming fleet configuration based on wireless power routing optionsavailable for candidate robots. Examples include, without limitation asingle power pack for wirelessly providing power to on-robotsub-assemblies, such as an organ-on-chip sub-assembly being powered overa robot-local wireless power routing system. Wireless power routing andmanagement may be extended to removable robot sensor-likesub-assemblies, such as the organ-on-chip example, that may be deployedseparate from but within wireless power routing range of a robot. Thismay be useful for environments where the sensor and robot cannot beco-located (e.g., due to size, environmental, or other constraints).According to some example implementations, a fleet management platformhaving a control tower for combined control of robots, such as MPRs,SPRs and exoskeletons, and additive manufacturing systems may also havean artificial intelligence system for automated design and 3D printingof robotic accessories. In some of these examples, the artificialintelligence system may automate design and 3D printing based oncontextual task recognition. This task recognition may rely on use ofshape recognition sensors (e.g., vision sensors) and operating history(for the robot or based on another factor, such as the task) todetermine, for example robot end effector requirements for completingthe task. In embodiments, a result of this AI-generated task recognitionmay be provided to the control tower to further enhance flexibleon-demand additive manufacturing based on recognition of a task to beperformed. In embodiments, the control tower may further combine robotcontrol of 3D printing of contextually-determined end effectors withcontrol of robotic 3D printing for additive manufacturing, therebyincreasing further the value of a 3D printing capability of a fleet ofrobots. In embodiments, such a combination may facilitate fieldmaintenance of robots, production equipment, warranty repairs and thelike. Yet further, use of artificial intelligence to facilitate taskrecognition may improve autonomous responsiveness for production systemservice/repair where some details of the required task may be unknown(e.g., fully automated production operations) until a robot is present.

According to some example implementations, a robot fleet managementplatform having autonomous local system task assignment adaptivity basedon sensed local context may also be integrated with supply chaininfrastructure entities for enhanced dynamic supply chain adaptivity andefficiency. In some of these example implementations, application oflocal system task assignment adaptivity with supply chain integrationmay enhance capabilities of, for example, in-container deployed robots.This combination of fleet management capabilities may also facilitatecoordination among robots (e.g., based on peer communications and thelike) along a supply chain, such as those deployed in or with a smartcontainer and the like, thereby providing flexibility when configuringindividual robots ahead of time. In an example, a set of robots deployedwith a long haul truck, ship or the like may assign supply chain tasksamong themselves based on locally sensed context. A set of tasks to beperformed during a trans-oceanic journey that is part of a supply chainmay be adaptively assigned based on local temporal context, such aslocal weather conditions, and the like.

According to some example implementations, a robot fleet managementplatform having smart contract support capabilities for among otherthings, negotiated routing of robots, may also have an artificialintelligence-governed data pipeline for supporting remote robotmanagement. In embodiments, smart contract terms that are detectable asa function of robot operation may influence how an AI-governed datapipeline is managed. As an example, a data pipeline may be managed toensure that, for example, a robot achieves certain data pipelinerequirements (e.g., average and peak throughput while ensuring highpriority data signals meet delivery requirements to ensure worker,robot, and/or client security, safety and other concerns). However, sucha robot data pipeline may also be managed (e.g., through AI-governance)to ensure that data representative of smart contract terms (e.g.,timeliness of reply, up-time, and the like) may be accurately and timelytracked (optionally recorded, saved, and later delivered) for managingthe relevant smart contract. Within this context, configuring a datapipeline for one or more robots associated with execution of a smartcontract (e.g., to provide warranty services) may include configuringvalue chain network (VCN) infrastructure elements for updating statesrelevant to smart contract terms and conditions. An AI-based datapipeline governance system may, for example, optimize use of sensordetection packages on robots throughout a VCN so that data pipelinerequirements can be met. In an example, a set of robots workingcooperatively throughout a value chain network may have sensor packagesconfigured (e.g., optimized) differently depending on their relativeposition in the value chain network when smart contract terms arefactored into robot configuration. As another example, configuration andutilization of on-robot data storage may also be influenced by smartcontract terms so that certain data that is collected (e.g., throughrobot sensor packages and the like) is stored locally and optionallycurated/filtered prior to being delivered over a data pipeline to asmart contract control facility. In this example, data pipelineresources may be prioritized so that only substantive departures fromnormal for certain smart contract terms utilize the pipeline.AI-governance of a data pipeline may enable local evaluation of smartcontract-impacting sensed data and so long as information derived fromrobot operations regarding meeting a smart contract requirement remainswithin an acceptable range, data pipeline resources are not required.

According to some example implementations, a robot fleet managementplatform having an artificial intelligence (AI) based robotic healthmonitoring system may also have hydraulic flow and actuation systemsthat are optimized for reducing hydraulic interconnections throughapplication of 3D printing in an additive manufacturing environment. Insome of these example implementations, information gleaned by theAI-based robotic health monitoring system may be directly applied tomitigating the likelihood of hydraulic interconnection failure by, forexample applying automated design and additive manufacturing to replace,such as during a preventive maintenance phase, multiple interconnectswith few or no interconnections. In embodiments, robotic healthmonitoring systems, such as computer vision systems for identifyingvisual defects or risks (e.g., identifying a hydraulic system with aplurality of interconnections), vibration-based detection (e.g.,identifying a hydraulic interconnect sub-assembly that is beingsubjected to fault-inducing levels of vibration), temperature sensingsystems that can provide thermal data about hydraulic system components(e.g., interconnections and the like) to influence which portions of amultiple interconnect hydraulic system are better candidate for use ofadditive manufacturing approaches to reducing failure risk of suchhydraulic systems. In embodiments, the AI-based robotic healthmonitoring system may further predict areas of failure, such ashydraulic interconnects that may be used as additive manufacturingrequirements for delivering hydraulic systems that are likely to be morerobust. Further, failure prediction capabilities may be used as acontrol for what components should be prioritized to be produced with anadditive manufacturing system. Yet further, scheduling and routing ofrobotic systems with additive manufacturing capabilities may beinfluenced by prediction capabilities of an AI-based robot healthmonitoring system, so that service or maintenance visit value can beoptimized by ensuring that additive manufacturing resources are eitherrouted to the service area for localized part manufacturing or areutilized to produce components (e.g., hydraulic assemblies with fewerinterconnections) so that they are available locally when a service caninclude deployment of improved reliability robotic elements.

According to some example implementations, a robot fleet managementplatform having an artificial intelligence-based shape recognitioncapability for automated task execution may also have a system forcoordinated control of robotic systems that incorporate 3D printing fortask execution. A robotic sensing and analysis system may use AI toanalyze visual images and sensor information along with past operatinghistory and task criteria (e.g., definition, objectives, and the like)to evaluate an object associated with a task, such as an object uponwhich a robotic operation is to be performed. The object analysis mayfacilitate determining one or more operations for performing an assignedtask, optionally including a type of end effector or other physicalinterface required to perform the task given the analysis. Inembodiments, the one or more operations required may include selectionand use of an particular type of end effector, such as a gripper,j-hook, pressure sensitive clamp, grip and rotate capability and thelike. In embodiments, the 3D printing control capability of the robot orof a companion robot configured to facilitate performing the task may beutilized to produce a suitable end effector, adaptor, or other featurebased on the visual and/or sensed analysis associated with the object.In an example, an object may have a keyhole type interface for handlingthe object. The image analysis may detect this feature of the object andcommission the 3D printing control system to produce a key suitable foruse with the object. Another example of combining robotic object sensing(e.g., shape recognition and the like) and control of 3D printingcapabilities for executing one or more operations of a task associatedwith an object includes sensing a shape of an object beingnon-rectilinear (e.g., round, oval, oblong) with no discernable flatsurface. The artificial intelligence-based shape recognition mayfacilitate detecting a suitable orientation for lifting the object,including a shape and size of contact surface required. This contactsurface shape and size information may be provided to the 3D printingcontrol system to produce an adaptor for an armature of the robot. Aresult of the AI-based shape recognition may identify the object assimilar to a type that was previously encountered by the platform. As anexample, parameters of the object may be used to identify candidateobjects in a library of objects for which the platform has managed afleet of robot tasks. The library may further indicate that a sling wassuccessfully used on one or more previous encounters with this class ofobject. In embodiments, the control of robotic system for 3D printingmay be directed to produce a suitable sling to be used by one or morerobots assigned to perform the object-specific task to lift andtransport the object. In yet another exemplary embodiment of a robotmanagement platform having both 3D printing control and artificialintelligence-based shape recognition capabilities, repair of the objectmay be achieved by use of visual and other sensors of the robotic systemdetermining that a handle of the object to be repaired is broken,thereby preventing performance of the repair as instructed. Based on thedetermination of this unexpected condition, a supplemental set of robotoperations may be generated for the current repair assignment toinstruct the robotic control system for 3D printing to fashion areplacement handle or perform a repair of the handle (e.g., mend a breakin a structural portion of the handle). These supplemental operationsmay be determined based on an assessment of an object to be repaired andintegrated in the current instance of the object repair process evenwhen the cause of failure that requires the repair task is other thanthe handle.

According to some example implementations, a robot fleet managementplatform having a conformable (e.g., liquid) lens vision system may alsohave an AI loop-based training and learning system that may be focusedon completing a set of tasks using quality of task completion as one ofone or more training factors. In embodiments, a conformable lens visionsystem may be configured, controlled, and adapted through use ofartificial intelligence for improving image formation. Feedback from theAI loop-based training and learning system may be used as one element offeedback for adjusting the conformable lens for improved imageformation. In embodiments, a combined AI system may facilitate adaptingthe conformable lens to improve quality of task completion. Factors suchas breakage of task objects and/or robotic components, when based onrobot operations that have a track record of success (e.g., not breakingthings) may suggest that image formation needs improvement. A roboticvision system with conformable lens technology may further improve robotoperations by using the loop-based learning capabilities to train itselfto detect and provide guidance to avoid task execution risk factors,such as objects along a path, and the like.

According to some example implementations, a robot fleet managementplatform having quantum optimization of thermal and energy factors in arobotic system may also have chip-sensor system (e.g., organ on a chipand the like) that provided biological sensing and evaluation. In someof these embodiments, the system senses radioactivity for evaluatingconditions associated with use and deployment of radioactive materials(e.g., as a fuel for an electricity generator). Sensitivity ofradioactive sensors and many other types of sensors may be impacted bytemperature conditions proximal to the sensing element. Ensuring thatthermal factors are automatically and properly addressed throughout arobot task assignment and over the life of the robot (or at least thesensing element) may improve sensitivity and therefore potentiallyfacilitate detecting potentially dangerous levels of radioactivity witha greater margin of safety. Maintaining thermal stability may furtherprovide benefits to other robotic sensing capabilities, such aschip-based medical diagnosis sensors, chip-based medical laboratorytesting, and the like.

According to some example implementations, a robot fleet managementplatform having a computer vision infrastructure for tracking andgoverning general robotic assets may also have shared economy roboticresource scheduling and routing capabilities. In embodiments, thecomputer vision robot tracking infrastructure may provide contextualdata to an autonomous robotic resource routing embodiment of the sharedeconomy robotic resource scheduling and routing capabilities. In anexample, the computer vision infrastructure may detect out-of-compliancerobotic behaviors that may indicate a need for routing of roboticresources to replace/support/regulate one or more robots at the sourceof the detected out-of-compliance behaviors. Further a computer visioninfrastructure for governing robotic assets may provide evidence of taskcompletion for an autonomously routed robotic resource to facilitateautomated billing for deployment of and task completion by the routedresource. This evidence may further substantiate claims by third-parties(e.g., other robotic fleet platforms) of lack of required on-locationrobotic support, which may include lack of on-location presence,out-of-compliance robotic behaviors and the like.

FIG. 143 illustrates an example data flow of the MPR 12100 adapted toharvest crops of produce. In embodiments, the data flow of the MPR 12100is executed in part by the vision and sensing system 12112, a motionplanning system 12158, a robot control system 12150, and a modulemanagement system 12148 of the MPR 12100. In this example, the MPR 12100may be employed as part of a robotic fleet that services an agriculturalenvironment (e.g., an outdoor farming facility, an indoor farmingfacility, a container configured for growing produce, or the like),whereby the MPR 12100 selectively harvest agricultural units. Inembodiments, the MPR 12100 may be configured to identify the units inthe agricultural farm that are ready to be harvested. In response, theMPR 12100 navigates the environment to reach such units to perform theharvesting task. The MPR 12100 may utilize vision and sensing system12112 for identifying which units are ready to be harvested. Inresponse, the motion planning system 12158 may create a motion plan tonavigate the environment and reach the location of the units determinedto be ready to be harvested. In embodiments, the MPR 12100 may selectone or more suitable end effectors 12124 from module system 12120 forselective harvesting of the ready unit.

At 12652, the MPR 12100 may capture image data from an agriculturalenvironment. The image data may include the one or more images capturedby cameras and/or other image sensors of the vision and sensing system12112. In some embodiments, the one or more of the images may becaptured using a camera 12608 with the conformable variable focus liquidlens 12612. The images may be captured from multiple differentviewpoints and may include one or more aerial images. In embodiments,the camera and/or other image sensors may be integrated in the housingof the MPR 12100, such that the MPR 12100 navigates the agriculturalenvironment to capture the images. Additionally or alternatively, one ormore cameras and/or other image sensors may be integrated in otherrobots or may be positioned in various areas of the agriculturalenvironment, such that the captured images are communicated to the MPR12100 for processing. The image data may be provided to the machinevision system 12618 of the intelligence layer 12140.

At 12654, the machine vision system 12618 may then analyze the imagedata to identify obstacles in the environment. In embodiments, themachine vision system 12618 may leverage one or more neural networkmodels (e.g., a CNN or RCNN) to detect various objects and obstacles inthe images. In embodiments, the motion planning system 12158 builds amotion planning graph representing the geometric structure of theenvironment as well as the different possible paths that may lead to thecrops to be harvested (target objects).

At 12656, the motion planning system creates a motion plan for the MPR12100 to identify an optimal path therefor. In embodiments, the motionplanning system 12158 works with the intelligence layer 12140 todetermine an optimal path after taking in account a cost function basedon collision assessment. The optimal path may be communicated to therobot control system 12150. At 12658, the robot control system 12150drives the actuators in the transport system 12110 enabling the MPR12100 to navigate to the location of the units to be harvested. Thenavigation actions for MPR 12100 (e.g., move forward, move backward,turn right, rotate and the like) are based on a trained navigationpolicy (machine learning algorithm). At 12660, a reinforcement learningsystem of 12622 the intelligence layer 12140 may collect the outcomedata for the navigation policy so as to update and improve the policy.

Once the MPR 12100 is in the proximity of the crops to be harvested, thecontroller may drive the movable arm to click additional images usingthe camera 12608 mounted at the end of the arm. The movable arm must beflexible in a dynamic environment and accurate enough not to damage thecrops while moving.

At 12662, the MPR 12100 may capture additional image data. For example,a camera 12608 and/or other sensors mounted on the arm of the MPR 12100may capture images or other sensor data an area proximate to the arm ofthe MPR 12100. At 12664, the machine vision system 12618 analyzes theimages (using neural network models including CNN or RCNN) to identifythe one or more crops to be harvested. At 12666, the images may beanalyzed by the motion planning system 12158 to determine the optimalpath as well as build a motion plan for the robot arm so as to rotate ormove the arm without damaging the crops. At 12668, the optimal path maybe communicated to the control system 12150 and the controller 12160 maydrive actuators in the arm and end effectors 12124 to harvest the unitsof crops. At 12670, the outcome data for the harvesting policy iscollected by the reinforcement learning system 12622 in the intelligencelayer 12140 as feedback to improve the harvesting policy.

Smart Containers

In some embodiments, a value chain network may include a smartintermodal shipping container system 13000 that enables variouscapabilities noted above, including smart container fleet managementservices within a value chain network. The system 1300 (such term, asnoted above, encompassing, except where context indicates otherwise,systems, methods, articles of manufacture, devices, machines, equipment,algorithms, parts, components, services, modules, workflows, processes,structures, products, and other elements) is arranged to embody orenable a range of highly functional, smart shipping containers, as wellas to configure fleets of smart container operating units to connect,engage and coordinate with other elements and entities of a value chainnetwork, such as to perform freight storage and/or transportationservices in an improved manner, to coordinate operations with factories,ships, loading docks, ports, warehouses, and transportationinfrastructure (such as trucking, railways and the like), to undertake(autonomously, under remote control, or by a combination) in-containeroperations (such as packaging, finishing, manufacturing, movement orarrangement of items and/or handling of storage conditions) and/or toundertake autonomous or remotely controlled mobility (or a combinationthereof); among other capabilities. In embodiments, smart containers mayphysically store and transport cargo, wherein cargo may refer to anycommodities, merchandise goods, materials, liquids, solids, powders,gases, foods, and many others.

In embodiments, the intermodal smart containers 13026 may includecontainers of many different types, classes, sizes, weights, materials,shapes, capabilities, or the like. In embodiments, the smart containersmay include standard rectangular containers, such as ones havingdimensions of 8-ft wide by 20-ft or 40-ft long, or non-standardcontainers. In embodiments, the smart containers may include 40-fthigh-cube containers, 45-ft high-cube containers, 48-ft high-cubecontainers, 53-ft high-cube containers, or the like, as well as anyother size container that, in certain preferred embodiment, has beendesigned by supply chain and transportation operators for compatibilitywith infrastructure elements of docks, factories, ports, trucks, trains,or the like. In embodiments, the smart containers may be tank containers(e.g., for liquids, gases, solids, powders, or the like),general-purpose dry vans (e.g., for boxes, cartons, cases, sacks, bales,pallets, drums, or the like), rolling floor containers, garmentainers(e.g., for shipping garments on hangers), ventilated containers(passively or actively ventilated), temperature-controlled containers(e.g., insulated, refrigerated, and/or heated), bulk containers,open-top containers, open-side containers, log cradles, platform-basedcontainers (e.g., flat rack and bolster containers), rotating and/ormixing containers (e.g., cement mixers), aviation containers (unit loaddevices), automotive containers (e.g., for moving passenger vehicles),bioprotective containers (for shipping toxic or bioactive materials,such as involving positive or negative pressure systems to regulateairflow), and many others. In some embodiments, smart containers may besmart packages (e.g., a small 16-in by 12-in by 12-in box and many othersmaller and larger sizes). In some embodiments, smart containers may beembodied as, include, integrate with, or use robots that are configuredto manipulate, transport, store or deliver a payload. In embodiments,smart containers may include mechanisms to enable expanding orretracting external or internal walls, housing elements, or otherinternal elements, such as to increase or decrease the volume of thecontainer or to vary the dimensions of one or more partitions of thespace within the container. The smart containers may haveself-assembling and/or self-disassembling mechanisms. In embodiments,the smart containers may be in the shape of a rectangular solid, a cube,sphere, a cylinder, or other shape, including in embodiments shapes thatinclude linear, non-linear, and irregular forms, such as organic-like orbiomimetic forms (such as to enable thermal management by providing aconductive or convective interface to a heating or cooling environmentor active heating or cooling element).

The smart container 13026 may consist of or include various sets ofmaterials, such as corrugated weathering steel, steel alloys, stainlesssteel, aluminum, cast iron, concrete, ceramic material(s), other alloys,glass, other metals, plastics, plywood, bamboo, cardboard, wood, and/ormany other materials. In embodiments, the smart containers may bebiodegradable. In embodiments, the smart containers may be 3D printedsmart containers or may contain 3D-printed elements. In embodiments, the3D printed smart containers may be printed as single integrated units.In embodiments, the 3D printed smart containers may have embedded 3Dprinted electronics. In embodiments, a smart container contains a 3Dprinter or other additive manufacturing facility that prints one or morecomponents, tools, accessories or the like for integration into and/oruse by the container.

In embodiments, the intermodal smart container 13026 may be autonomousand/or self-driving. For example, an intermodal smart container may beconfigured to autonomously traverse terrestrial, subterrestrial, marine,sub-marine, air, and/or space environments. In embodiments, the smartcontainer may be configured with retractable or non-retractable wheels(e.g., for roadways, terrain, rail, or the like), continuous tracksystems, skis, sails, propellers, propulsion systems, legs, and the liketo support transportation in different environments. Additionally, oralternatively, the smart container may be transported by traditionalmethods of railways, trucks, container ships, air, and the like. Forexample, a smart container with retractable wheels may be able toautonomously drive across a container terminal, drive up a ramp onto acontainer ship, drive to a particular location on a container ship, andbe transported via the container ship to another container terminal.Smart containers may also be configured to be transported by hyperloopsystems and networks.

In some embodiments, the smart containers 13026 may be configured to beself-stacking. For example, the smart containers may have mechanicalstacking rails on the sides that allow containers to slide up and downand/or across other containers. In embodiments, the rails may beelectromagnetic rails. Additionally, or alternatively, smart containersmay be configured with retractable or non-retractable mechanical liftsystems, container handling devices, and the like. For example, smartcontainers may be configured with lift systems, such as scissor lifts(e.g., hydraulic scissor lifts, diesel scissor lifts, electric scissorlifts, rough terrain scissor lifts, or pneumatic scissor lifts),retractable extendable legs, or the like that enable the self-stackingof smart shipping containers. In another example, smart containers maybe configured as cranes, forklifts, reach stackers, or the like.

In embodiments, the smart intermodal container system 13000's fleetmanagement system 13002 receives a freight storage and/or transportationservice order (e.g., from a client device) and identifies the freightstorage and/or transportation services to be performed in completion ofthe order. In some embodiments, a user may be presented a GUI on aclient device to provide one or more freight transportation servicerequirement parameter values. For instance, the GUI may include fieldsfor the user to define service timing requirements (e.g., how quicklycargo needs to be delivered), origin of shipment, whether the shipmentis received at a terminal/ramp or at another location, destination ofshipment, whether the delivery is to a terminal/ramp or to anotherlocation, type of container required, number of containers required,container usage requirements (e.g., full container (FCL) vs. sharedcontainer (LCL)), container size requirements for FCL shipments (e.g.,20-ft container, 40-ft container, 40-ft high-cube, tank, or the like),cargo descriptions for LCL shipments (e.g., number of packages, totalvolume, total weight, or the like), whether the cargo includes personaleffects, and the like. In response to determining the requirements ofthe freight storage and/or transportation service order, the smartcontainer system's fleet management system 13002 may determine a smartcontainer fleet configuration that includes a set of smart containeroperating units and may assign smart container operating units to thefreight storage and/or transportation service order. As used herein, asmart container operating unit may refer to an individual smartcontainer, a team of smart containers, a fleet of smart containers thatoperate to complete a request, or the like. As will be discussed, insome embodiments, the smart container fleet management system 13002 maydefine a configuration of one or more smart containers to execute afreight storage and/or transportation service order and/or to operate ina certain type of environment as part of the fleet configuration. Aswill be discussed, a smart container may be configured with variousmodules that allow the smart container to perform certain tasks. Forinstance, a smart container may be provisioned with specialized devicesand systems, such as IoT devices (including cameras and sensor-baseddevices), edge network devices, chipsets, chips and other devices thathave data storage, computation, processing and/or connectivitycapability to enable the smart container to perform intelligence tasks;specialized sensors for particular environments; liquid lenses forenabling certain machine-vision functionality; specialized robots thatperform certain tasks, specialized tools and/or systems that are taskspecific (e.g., lighting, irrigation systems, heating, cooling, 3Dprinter, clamps, grippers, drills, lifts, cranes, conveyors, mixers,forklifts, and/or the like); smart ramps and the like; and/or othermodules that configure the smart container to perform a certain task orset of tasks.

In some embodiments, the smart intermodal container system's fleetmanagement system 13002 may define a set of workflows, wherein aworkflow may define an order by which certain freight storage and/ortransportation services or tasks are performed and the smart containeroperating unit(s) that is/are assigned to the respective service ortask. In some embodiments, the smart container fleet management systemmay perform workflow simulations to iteratively redefine fleetconfigurations and/or workflows to substantially optimize the operationof the smart container fleet. For example, the fleet configurationsand/or workflows may be iteratively adjusted to reduce costs, improvelogistical efficiencies, reduce the overall shipment time, reduce carbonemissions, or the like. Once the fleet configuration and workflows arefinalized, the smart container fleet management system may deploy thefleet. In some embodiments, the smart container fleet management systemmay facilitate the logistics involved with supplying smart containeroperating units and/or smart container components, and/or supportingresources. Furthermore, in some embodiments, the smart container fleetmanagement system may leverage additive manufacturing capabilities, suchas 3D printers or other capabilities described herein, in furtherance ofaccommodating localized preferences with rapid customization, such thatitems that are capable of being 3D-printed within the smart container asit approaches a destination. In embodiments, the smart container fleetmanagement system may monitor the smart container fleet, including thestatus of smart container operating units, the performance of smartcontainer units (e.g., timing performance, financial performance, or thelike), the status of cargo contained within the smart container, and thelike. In some of these embodiments, the smart container system 13000 mayautomate maintenance of smart containers and/or resources to ensure anefficient use of an available inventory and/or to reduce downtime.

In embodiments, the smart container system 13000 may automaticallygovern the storage conditions of a set of items, such as by classifyingthe items (such as using a machine vision-based artificial intelligencesystem that is trained on a training set of data to recognize items bytype); determining the appropriate storage conditions for the set (whichmay include factoring in, such as based on an economic or other model,the value of items that may have different optimal storage conditions,which may also be undertaken by an artificial intelligence system, suchas one trained on expert interactions with a model, expert instructionsor setting of storage conditions and/or outcomes from operations, amongothers); and providing an instruction set for storage, such as a set oftemperature profiles, humidity profiles, movement profiles (e.g., toavoid excessive shaking of fragile goods) and the like, which may betaken as a recommendation for further consideration (such as by a humanoperator or other entity) or used an input to an autonomous orsemi-autonomous control system for the environment. An AI or expertsystem may include a variety of factors in determining appropriatestorage, including factors related to the sensitivity of contents toshaking, temperature variations, humidity, radiation, chemical factors(e.g., corrosion from salt), and many others; factors related to valueof contents (such as the price, cost, or profit margins of contents,including based on market factors, contractual terms, risk allocation,and the like); and transportation factors (such as equipment conditions,road, waterway, airway or railway conditions, weather conditions, andthe like); and others. Over time, input factors may be added or removeddepending on the success of artificial intelligence systems (includingmodel-based, deep learning, or other systems), which may be seeded withexpert models, trained on expert-labeled data sets and/or upon expertinteractions (such as using robotic process automation) and/or trainedon outcomes from use. Thus, a self-governing or autonomous storagecondition system, or a semi-autonomous storage condition system, may beintegrated with or into a container system.

In some embodiments, the smart container system may support digitaltwins that depict the status of the smart container operating unitsand/or performance based on data received from the smart containeroperating units or other suitable data sources, such as edge devices,environmental sensor systems, logistics systems, and/or other suitabledata sources. The digital twins served by the smart intermodal containersystem may be adapted for various uses. In some example embodiments, adigital twin may be configured to provide a status of a smart containerfleet, including individual smart containers within the fleet. In theseexamples, a user may drill down onto individual smart containers in ateam or fleet of smart containers to view the status of the smartcontainers. For example, the user may view the battery life of a smartcontainer, the availability of smart container energy sources and/orcharging stations, the location of a smart container, the mobilityoptions for the smart container, the status of cargo within the smartcontainer, task completion status of a smart container, maintenancealerts of a smart container, and/or the like. In some exampleembodiments, the smart container system may serve environment digitaltwins that depict the environment of a smart container fleet withreal-time information, such as locations of available smart containerinfrastructure and/or modes of transport (e.g., container ships,submarines, spacecraft, hyperloops, railways, trucks, or the like),facilities (e.g., container terminals, shipyards, storage areas, or thelike), objects, other smart containers, sensor readings of theenvironment, and the like. In these embodiments, a user may leverage anenvironment digital twin to provide remote control commands to a smartcontainer, a team of smart containers, or a fleet of smart containers.For instance, a smart container or team of smart containers mayencounter an unidentified object obstructing a route and may need togenerate a decision related to re-routing. In some embodiments, thesmart container fleet management system may obtain relevant data (e.g.,LIDAR data, video feeds, environment maps, and the like) which may bedepicted in an environment digital twin. The user may view the currentscenario in the environment digital twin and may provide instructions tothe smart container fleet on how to proceed given the scenario presentedin the environment digital twin. The foregoing are non-limiting examplesof digital twins that may be used in connection with a smart containerfleet management system and other examples are discussed below.

FIG. 144 illustrates an example environment of a smart container system13000. In embodiments, a smart container system 13000 includes a fleetmanagement system 13002 a data processing system 13024, and anintelligence service 13004 (e.g., a system level intelligence service13004). In embodiments, the fleet management system 13002 configures andmanages smart container operating units 13040 and/or freight storageand/or transportation services and/or tasks that are performed by smartcontainer operating units 13040. As will be discussed, a smart containeroperating unit 13040 may refer to individual smart containers,individual smart container task assemblies 13050, smart container fleets13060, and/or smart container fleet support units 13080.

In embodiments, the fleet management system 13002 includes, but is notlimited to, a communication management system 13010, the remote-controlsystem 13012, a resource provisioning system 13014, a logistics system13016, a job configuration system 13018, a fleet configuration system13020, an order execution, monitoring, and reporting system 13022 (alsoreferred to as an “order execution system” 13022), a human interfacesystem 13038, and a maintenance management system 13028. In embodiments,the communication management system 13010 is configured to facilitatefleet management system communications, including with elements externalto the smart container system 13000. In embodiments, the fleetmanagement system communications include satellite communications. Inembodiments, the remote-control system 13012 is configured to manage andenable control of smart container operating units and fleet resourcesremotely. In embodiments, the resource provisioning system 13014 isconfigured to handle allocation and access to fleet resources (e.g.,smart container operating units). In embodiments, the logistics system13016 coordinates use and transportation of fleet resources and suppliesto smart container operating units. In embodiments, the maintenancemanagement system 13028 facilitates coordinated, timely maintenance offleet resources. In embodiments, the job configuration system 13018generates an order execution plan based on a freight storage and/ortransportation service order. In embodiments, a fleet configurationsystem 13020 configures smart container operating units (e.g.,individual smart containers and/or smart container fleets) to completean order execution plan. In embodiments, the order execution system13022 executes, monitors, and/or reports on freight storage and/ortransportation services being performed by smart container operatingunits (e.g., in accordance with an order execution plan) to ensureefficient use of fleet resources while executing the order executionplan and addressing shipment and fleet related reporting requirements.In embodiments, the human interface system provides an interface bywhich a human user may interface with a smart container operating unit.

As mentioned, a smart container operating unit 13040 may refer toindividual smart containers 13026, individual smart container taskassemblies 13050, smart container fleets 13060, and/or smart containerfleet support units 13080.

As shown in FIG. 145 , smart container 13026 may include a baselinesystem 13106, a smart container control system 13104, and a smartcontainer security system 13046. In embodiments, the smart containercontrol system 13104 includes a data processing system 13024 and anintelligence service 13004. As will be discussed, the data processingsystem may include data processing resources that may be centralizedand/or distributed amongst a team or fleet of smart containers.Additionally, or alternatively, the data processing resources mayinclude general purpose chipsets, specialized chipsets, and/orconfigurable chipsets. As will be discussed, the intelligence service13004 performs intelligence related tasks on behalf of the smartcontainer or a collection of smart containers (e.g., a task assembly orfleet). For example, the intelligence service 13004 may perform suchtasks as artificial intelligence, machine-learning, natural languageprocessing, machine vision, analytics, and/or the like and may leveragecomplex data structures (e.g., digital twins) and disparate data sources(e.g., from IoT, edge and other network-enabled devices, fromon-premises and cloud-deployed databases and other resources, and/orfrom APIs, event streams, logs, or other data sources, among manyothers) in performance thereof. Smart container-level and fleet-levelintelligence services are discussed in greater detail below. Inembodiments, the smart container security system 13046 performs securityrelated functions on behalf of a smart container or a collection ofsmart containers (e.g., a task assembly or fleet). Thesesecurity-related functions may include autonomous adaptive andnon-adaptive security functions as well as manual security functions.

In embodiments, a baseline system 13106 of a smart container 13026 mayinclude an energy storage and power distribution system 13112, enclosure13114, an electro-mechanical and/or electro-fluidic system 13116, atransport system 13118, a vision and sensing system 13120, and/or astructural system 13122. As will be discussed further below, theconfiguration of a baseline system of a smart container 13026 depends onthe types of freight storage and/or transportation services and/or tasksthat the smart container 13026 is configured to perform and/or the typeof environments that the smart container is intended to operate in. Forexample, smart containers that are configured to operate in deep waterconditions may have different baseline systems than smart containersthat are configured to operate in arctic conditions or aerial smartcontainers.

In embodiments, a smart container 13026 may further include a modulesystem 13102 that allows the smart container to be configured withvarious hardware and/or software components. In this way, the smartcontainer 13026 may be fitted with different accessories, sensor sets,chipsets, motive adaptors, and/or the like depending on the range offreight storage and/or transportation services and/or tasks that thesmart container is configured for and/or the environments the smartcontainer is configured to operate in. In embodiments, the module system13102 may include control module interfaces 13108 and physical moduleinterfaces 13110. The control module interfaces 13108 and physicalmodule interfaces 13110 may refer to mechanical, electrical, and/ordigital interfaces that receive auxiliary components to configure asmart container 13026 to perform certain tasks. In embodiments, thecontrol module interfaces 13108 receive (or otherwise “connect” to)auxiliary components that alter one or more features that relate tocontrol of smart container 13026. These may include chipsets (e.g., AIchipsets, machine-learning chipsets, machine-vision chipsets,communications chipsets, and/or the like), sensor modules, communicationmodules, AI modules, security modules, computing modules, and/or thelike. In embodiments, the physical module interfaces 13110 receive (orotherwise connect to) auxiliary physical modules that alter the physicalactions that may be taken by the smart container 13026 and/or thephysical operation of the smart container. Examples of physical modulesmay include, but are not limited to, wheels, robotic arms, sortingsystems, packaging systems, 3D printers, cranes, lifts, power supplies,and/or the like. As will be discussed, a smart container 13026 may bereconfigured to perform one or more tasks in completion of a freightstorage and/or transportation service order. In these embodiments, thesmart container system 13000 may define an order execution plan and asupporting smart container or smart container fleet and may provisionone or more modules to a smart container 13026, such that the smartcontainer 13026 is reconfigured to perform one or more specified tasksin the order execution plan.

Referring back to FIG. 144 , individual smart container task assemblies13050 may refer to a collection of one or more individual smartcontainers that are assigned to a freight storage and/or transportationservice order. The smart containers in a smart container task assemblymay include any combination of smart containers (e.g., a 40-ft smartcontainer and a tanker). In some embodiments, an individual smartcontainer task assembly 13050 may include a local manager that controlsor otherwise provides instructions to smart containers in the taskassembly 13050. In these embodiments, the local manager may be adesignated supervisor smart container, a robot, or a human operator. Inembodiments, the smart container supervisor may act as an edge device onbehalf of the task assembly 13050, such that the smart containersupervisor may be allocated specific processing and/or communicationcapabilities that allow the smart container supervisor to communicatewith the smart container system 13000 or other suitable devices orsystems and/or to perform data processing operations on behalf of thetask assembly 13050. In embodiments, a smart container fleet is acollection of individual smart containers and/or task assemblies thatcollectively perform a set of tasks in completion of a freight storageand/or transportation service order. Furthermore, fleets may be arrangedas a fleet of task groups, regional fleets, and/or a fleet of fleets. Inembodiments, a smart container fleet may be supported by smart containerfleet support. In embodiments, examples of smart container fleet supportmay include on premises edge and IoT devices, local data storages (andcorresponding data interfaces), forklifts, cranes, mechanical lifts,reach stackers, maintenance support, charging stations and devices,replacement parts, batteries, accessories, docking stations, spareparts, and/or technicians.

In embodiments, the smart container system 13000 may include a dataprocessing system 13024. In embodiments, the data processing system13024 includes a data handling service 13032 and a data processingservice 13030. The data handling service 13032 is configured to store,retrieve, and otherwise manage the data of the smart container system13000. In embodiments, the data handling service 13032 accesses a set ofdata stores 13042 and/or libraries 13044, whereby the data handlingservice 13032 writes and reads data from the data stores 13042 and/orlibraries 13044 on behalf of other components of the smart containersystem 13000 and/or the smart container operating units 13040. Inembodiments, the data processing service 13030 performs data processingoperations on behalf of other components of the smart container system13000 and/or the smart container operating units 13040. For example, thedata processing service 13030 may perform database operations (e.g.,table joins, retrieves, etc.), data fusion operations, and the like. Inembodiments, the data processing system may include distributedresources, centralized resources, and/or “on-chip” resources.

As shown in FIG. 146 , an intelligence service 13004 is adapted toprovide intelligence services to the smart intermodal container system13000 and/or other intelligence service clients. In some embodiments,the intelligence service 13004 framework may be at least partiallyreplicated in the smart intermodal container system 13000, smartcontainers, VCN control towers, various VCN entities, and/or otherintel. In these embodiments, the smart intermodal container system 13000may include some or all of the capabilities of the intelligence service13004, whereby the intelligence service 13004 is adapted for thespecific functions performed by the subsystems of the intelligenceclient. Additionally, or alternatively, in some embodiments, theintelligence service 13004 may be implemented as a set of microservices,such that different intelligence service clients may leverage theintelligence service 13004 via one or more APIs exposed to theintelligence clients. In these embodiments, the intelligence service13004 may be configured to perform various types of intelligenceservices that may be adapted for different intelligence service clients.In either of these configurations, an intelligence service client 13324may provide an intelligence request to the intelligence service 13004,whereby the request is to perform a specific intelligence task (e.g., adetection/identification, a decision, a recommendation, a report, aninstruction, a classification, a prediction, an optimization, a controlaction, a configuration action, an automation, a training action, an NLPrequest, or the like). In response, the intelligence service 13004executes the requested intelligence task and returns a response to theintelligence service client 13324. Additionally, or alternatively, insome embodiments, the intelligence service 13004 may be implementedusing one or more specialized chips that are configured to provide AIassisted microservices such as image processing, diagnostics, locationand orientation, chemical analysis, data processing, and so forth. Forexample, a smart container having an AI chipset may be configured toimplement the intelligence service 13004 to provide navigationinstructions, container security (e.g., biometric access), environmentalmonitoring, generate insights related to container and/or cargo weight,generate insights related to container capacity, generate insightsrelated to the container's structural integrity, generate insightsrelated to cargo damage, predict regulatory issues (e.g., customs),detect illegal and/or dangerous cargo, provide autonomous control, orthe like.

It is further noted that in some scenarios, artificial intelligencemodules 13404 themselves may also be intelligence service clients. Forexample, a rules-based intelligence module 13428 may request anintelligence task from a machine learning module 13412 or a neuralnetwork module 13414, such as requesting a classification of an objectappearing in a video and/or a motion of the object. In this example, therules-based intelligence module 13428 may be an intelligence serviceclient 13324 that uses the classification to determine whether to take aspecified action. In another example, a machine vision module 13422 mayrequest a digital twin of a specified environment from a digital twinmodule 13420, such that the machine learning module 13412 may requestspecific data from the digital twin as features to train amachine-learned model that is trained for a specific environment.

In embodiments, an intelligence task may require specific types of datato respond to the request. For example, a machine vision task requiresone or more images (and potentially other data) to classify objectsappearing in an image or set of images, to determine features within theset of images (such as locations of items, codes or information on othercontainers (e.g., Bureau International des Containers (BIC) code, CSCapproval plates, ISO 6346 reporting marks, or the like), presence offaces, symbols or instructions, expressions, parameters of motion,changes in status, and many others), and the like. In another example,an NLP task requires audio of speech and/or text data (and potentiallyother data) to determine a meaning or other element of the speech and/ortext. In yet another example, an AI-based control task (e.g., a decisionon movement of a smart container) may require environment data (e.g.,maps, coordinates of known obstacles, images, and/or the like) and/or amotion plan to generate a decision related to the control of motion of asmart container. In a platform-level example, an analytics-basedreporting task may require data from a number of different databases togenerate a report. Thus, in embodiments, tasks that can be performed bythe intelligence service 13004 may require, or benefit from, specificintelligence service inputs 13470. In some embodiments, the intelligenceservice 13004 may be configured to receive and/or request specific datafrom the intelligence service inputs 13470 to perform a respectiveintelligence task. Additionally, or alternatively, the requestingintelligence service client 13324 may provide the specific data in therequest. For instance, the intelligence service 13004 may expose one ormore APIs to the intelligence service clients, whereby a requestingclient 13324 provides the specific data in the request via the API.Examples of intelligence service inputs may include, but are not limitedto, sensors that provide sensor data, video streams, audio streams,databases, data feeds, human input, and/or other suitable data.

In embodiments, the intelligence service 13004 may include anintelligence service controller 13402 and artificial intelligence (AI)modules 13404. In embodiments, the artificial intelligence service 13004receives an intelligence request from an intelligence service client13324 and any required data to process the request from the intelligenceservice client 13324. In response to the request and the specific data,one or more implicated artificial intelligence modules 13404 perform theintelligence task and output an “intelligence response”. Examples ofintelligence modules 13304 responses may include a decision (e.g., acontrol instruction, a proposed action, machine-generated text, and/orthe like), a prediction (e.g., a predicted meaning of a text snippet, apredicted outcome associated with a proposed action, a predicted faultcondition, and/or the like), a classification (e.g., a classification ofan object in an image, a classification of a spoken utterance, aclassified fault condition based on sensor data, and/or the like),and/or other suitable outputs of an artificial intelligence system.

In embodiments, artificial intelligence modules 13404 may include amachine learning module 13412, a rules-based module 13428, an analyticsmodule 13418, an RPA module 13416, a digital twin module 13420, amachine vision module 13422, an NLP module 13424, and/or a neuralnetwork module 13414. It is appreciated that the foregoing arenon-limiting examples of artificial intelligence modules, and that someof the modules may be included or leveraged by other artificialintelligence modules. For example, the NLP module 13424 and the machinevision module 13422 may leverage different neural networks that are partof the neural network module 13414 in performance of their respectivefunctions.

In embodiments, intelligence modules 13304 includes and provides accessto a machine learning module 13412 that may be integrated into or beaccessed by one or more intelligence service clients. In embodiments,the machine learning module 13412 may provide machine-based learningcapabilities, features, functions, and algorithms for use by anintelligence service client 13324 such as training machine learningmodels, leveraging machine learning models, reinforcing machine learningmodels, performing various clustering techniques, feature extraction,and/or the like. In an example, a machine learning module 13412 mayprovide machine learning computing, data storage, and feedbackinfrastructure to a simulation system. The machine learning module 13412may also operate cooperatively with other modules, such as therules-based module 13428, the machine vision module 13422, the RPAmodule 13416, and/or the like.

The machine learning module 13412 may define one or more machinelearning models for performing analytics, simulation, decision making,optimization, and predictive analytics related to data processing, dataanalysis, simulation creation, simulation prioritization, and simulationanalysis of one or more components or subsystems of an intelligenceservice client 13324. In embodiments, the machine learning models arealgorithms and/or statistical models that perform specific tasks withoutusing explicit instructions, relying instead on patterns and inference.The machine learning models build one or more mathematical models basedon training data to make predictions and/or decisions without beingexplicitly programmed to perform the specific tasks. In exampleimplementations, machine learning models may perform classification,prediction, regression, clustering, anomaly detection, recommendationgeneration, decision-making, optimization, and/or other tasks.

In embodiments, the machine learning models may perform various types ofclassification based on the input data. Classification is a predictivemodeling problem where a class label is predicted for a given example ofinput data. For example, machine learning models can perform binaryclassification, multi-class classification, or multi-labelclassification. In embodiments, the machine-learning model may output“confidence scores” that are indicative of a respective confidenceassociated with classification of the input into the respective class.In embodiments, the confidence scores can be compared to one or morethresholds to render a discrete categorical prediction. In embodiments,only a certain number of classes (e.g., one) with the relatively largestconfidence scores can be selected to render a discrete categoricalprediction.

In embodiments, machine learning models may output a probabilisticclassification. For example, machine learning models may predict, givena sample input, a probability distribution over a set of classes. Thus,rather than outputting only the most likely class to which the sampleinput should belong, machine learning models can output, for each class,a probability that the sample input belongs to such class. Inembodiments, the probability distribution over all possible classes cansum to one. In embodiments, a Softmax function, or other type offunction or layer can be used to turn a set of real values respectivelyassociated with the possible classes to a set of real values in therange (0, 1) that sum to one. In embodiments, the probabilities providedby the probability distribution can be compared to one or morethresholds to render a discrete categorical prediction. In embodiments,only a certain number of classes (e.g., one) with the relatively largestpredicted probability can be selected to render a discrete categoricalprediction.

In embodiments, machine learning models can perform regression toprovide output data in the form of a continuous numeric value. Asexamples, machine learning models can perform linear regression,polynomial regression, or nonlinear regression. As described, inembodiments, a Softmax function or other function or layer can be usedto squash a set of real values respectively associated with a two ormore possible classes to a set of real values in the range (0, 1) thatsum to one. For example, machine learning models can perform linearregression, polynomial regression, or nonlinear regression. As examples,machine learning models can perform simple regression or multipleregression. As described above, in some implementations, a Softmaxfunction or other function or layer can be used to squash a set of realvalues respectively associated with two or more possible classes to aset of real values in the range (0, 1) that sum to one.

In embodiments, machine learning models may perform various types ofclustering. For example, machine learning models may identify one ormore previously defined clusters to which the input data most likelycorresponds. In some implementations in which machine learning modelsperform clustering, machine learning models can be trained usingunsupervised learning techniques.

In embodiments, machine learning models may perform anomaly detection oroutlier detection. For example, machine learning models can identifyinput data that does not conform to an expected pattern or othercharacteristic (e.g., as previously observed from previous input data).As examples, the anomaly detection can be used for fraud detection orsystem failure detection.

In some implementations, machine learning models can provide output datain the form of one or more recommendations. For example, machinelearning models can be included in a recommendation system or engine. Asan example, given input data that describes previous outcomes forcertain entities (e.g., a score, ranking, or rating indicative of anamount of success or enjoyment), machine learning models can output asuggestion or recommendation of one or more additional entities that,based on the previous outcomes, are expected to have a desired outcome.

As described above, machine learning models can be or include one ormore of various different types of machine-learned models. Examples ofsuch different types of machine-learned models are provided below forillustration. One or more of the example models described below can beused (e.g., combined) to provide the output data in response to theinput data. Additional models beyond the example models provided belowcan be used as well.

In some implementations, machine learning models can be or include oneor more classifier models such as, for example, linear classificationmodels; quadratic classification models; etc. Machine learning modelsmay be or include one or more regression models such as, for example,simple linear regression models; multiple linear regression models;logistic regression models; stepwise regression models; multivariateadaptive regression splines; locally estimated scatterplot smoothingmodels; or the like.

In some examples, machine learning models can be or include one or moredecision tree-based models such as, for example, classification and/orregression trees; chi-squared automatic interaction detection decisiontrees; decision stumps; conditional decision trees; etc.

Machine learning models may be or include one or more kernel machines.In some implementations, machine learning models can be or include oneor more support vector machines. Machine learning models may be orinclude one or more instance-based learning models such as, for example,learning vector quantization models; self-organizing map models; locallyweighted learning models; etc. In some implementations, machine learningmodels can be or include one or more nearest neighbor models such as,for example, k-nearest neighbor classifications models; k-nearestneighbors regression models; etc. Machine learning models can be orinclude one or more Bayesian models such as, for example, naïve Bayesmodels; Gaussian naïve Bayes models; multinomial naïve Bayes models;averaged one-dependence estimators; Bayesian networks; Bayesian beliefnetworks; hidden Markov models; etc.

Machine learning models may include one or more clustering models suchas, for example, k-means clustering models; k-medians clustering models;expectation maximization models; hierarchical clustering models; etc.

In some implementations, machine learning models can perform one or moredimensionality reduction techniques such as, for example, principalcomponent analysis; kernel principal component analysis; graph-basedkernel principal component analysis; principal component regression;partial least squares regression; Sammon mapping; multidimensionalscaling; projection pursuit; linear discriminant analysis; mixturediscriminant analysis; quadratic discriminant analysis; generalizeddiscriminant analysis; flexible discriminant analysis; autoencoding;etc.

In some implementations, machine learning models can perform or besubjected to one or more reinforcement learning techniques such asMarkov decision processes; dynamic programming; Q functions orQ-learning; value function approaches; deep Q-networks; differentiableneural computers; asynchronous advantage actor-critics; deterministicpolicy gradient; etc.

In embodiments, artificial intelligence modules 13404 may include and/orprovide access to a neural network module 13414. In embodiments, theneural network module 13414 is configured to train, deploy, and/orleverage artificial neural networks (or “neural networks”) on behalf ofan intelligence service client 13324. It is noted that in thedescription, the term machine learning model may include neuralnetworks, and as such, the neural network module 13414 may be part ofthe machine learning module 13412. In embodiments, the neural networkmodule 13414 may be configured to train neural networks that may be usedby the smart container management system 13000 and other intelligenceservice clients. Non-limiting examples of different types of neuralnetworks may include any of the neural network types describedthroughout this disclosure and the documents incorporated herein byreference, including without limitation convolutional neural networks(CNN), deep convolutional neural networks (DCN), feed forward neuralnetworks (including deep feed forward neural networks), recurrent neuralnetworks (RNN) (including without limitation gated RNNs), long/shortterm memory (LTSM) neural networks, and the like, as well as hybrids orcombinations of the above, such as deployed in series, in parallel, inacyclic (e.g., directed graph-based) flows, and/or in more complex flowsthat may include intermediate decision nodes, recursive loops, and thelike, where a given type of neural network takes inputs from a datasource or other neural network and provides outputs that are includedwithin the input sets of another neural network until a flow iscompleted and a final output is provided. In embodiments, the neuralnetwork module 13414 may be leveraged by other artificial intelligencemodules 13404, such as the machine vision module 13422, the NLP module13424, the rules-based module 13428, the digital twin module 13420, andso on. Example applications of the neural network module 13414 aredescribed throughout the disclosure.

A neural network includes a group of connected nodes, which also can bereferred to as neurons or perceptrons. A neural network can be organizedinto one or more layers. Neural networks that include multiple layerscan be referred to as “deep” networks. A deep network can include aninput layer, an output layer, and one or more hidden layers positionedbetween the input layer and the output layer. The nodes of the neuralnetwork can be connected or non-fully connected.

In embodiments, the neural networks can be or include one or more feedforward neural networks. In feed forward networks, the connectionsbetween nodes do not form a cycle. For example, each connection canconnect a node from an earlier layer to a node from a later layer.

In embodiments, the neural networks can be or include one or morerecurrent neural networks. In some instances, at least some of the nodesof a recurrent neural network can form a cycle. Recurrent neuralnetworks can be especially useful for processing input data that issequential in nature. In particular, in some instances, a recurrentneural network can pass or retain information from a previous portion ofthe input data sequence to a subsequent portion of the input datasequence through the use of recurrent or directed cyclical nodeconnections.

In some examples, sequential input data can include time-series data(e.g., sensor data versus time or imagery captured at different times).For example, a recurrent neural network can analyze sensor data versustime to detect or predict a swipe direction, to perform handwritingrecognition, etc. Sequential input data may include words in a sentence(e.g., for natural language processing, speech detection or processing,etc.); notes in a musical composition; sequential actions taken by auser (e.g., to detect or predict sequential application usage);sequential object states; etc. In some example embodiments, recurrentneural networks include long short-term (LSTM) recurrent neuralnetworks; gated recurrent units; bi-direction recurrent neural networks;continuous time recurrent neural networks; neural history compressors;echo state networks; Elman networks; Jordan networks; recursive neuralnetworks; Hopfield networks; fully recurrent networks;sequence-to-sequence configurations; etc.

In some examples, neural networks can be or include one or morenon-recurrent sequence-to-sequence models based on self-attention, suchas Transformer networks. Details of an exemplary transformer network canbe found athttp://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf.

In embodiments, the neural networks can be or include one or moreconvolutional neural networks. In some instances, a convolutional neuralnetwork can include one or more convolutional layers that performconvolutions over input data using learned filters. Filters can also bereferred to as kernels. Convolutional neural networks can be especiallyuseful for vision problems such as when the input data includes imagerysuch as still images or video. However, convolutional neural networkscan also be applied for natural language processing.

In embodiments, the neural networks can be or include one or moregenerative networks such as, for example, generative adversarialnetworks. Generative networks can be used to generate new data such asnew images or other content.

In embodiments, the neural networks may be or include autoencoders. Insome instances, the aim of an autoencoder is to learn a representation(e.g., a lower-dimensional encoding) for a set of data, typically forthe purpose of dimensionality reduction. For example, in some instances,an autoencoder can seek to encode the input data and then provide outputdata that reconstructs the input data from the encoding. Recently, theautoencoder concept has become more widely used for learning generativemodels of data. In some instances, the autoencoder can includeadditional losses beyond reconstructing the input data.

In embodiments, the neural networks may be or include one or more otherforms of artificial neural networks such as, for example, deep Boltzmannmachines; deep belief networks; stacked autoencoders; etc. Any of theneural networks described herein can be combined (e.g., stacked) to formmore complex networks.

In embodiments, a neural network may include an input layer, a hiddenlayer, and an output layer with each layer comprising a plurality ofnodes or neurons that respond to different combinations of inputs fromthe previous layers. The connections between the neurons have numericweights that determine how much relative effect an input has on theoutput value of the node in question. Input layer may include aplurality of input nodes and that may provide information from theoutside world or input data (e.g., sensor data, image data, text data,audio data, etc.) to the neural network. The input data may be fromdifferent sources and may include library data x1, simulation data x2,user input data x3, training data x4 and outcome data x5. The inputnodes and may pass on the information to the next layer, and nocomputation may be performed by the input nodes. Hidden layers mayinclude a plurality of nodes. The nodes in the hidden layer and mayprocess the information from the input layer based on the weights of theconnections between the input layer and the hidden layer and transferinformation to the output layer. The output layer may include an outputnode, which processes information based on the weights of theconnections between the hidden layer and the output layer and isresponsible for computing and transferring information from the networkto the outside world, such as recognizing certain objects or activities,or predicting a condition or an action.

In embodiments, a neural network may include two or more hidden layersand may be referred to as a deep neural network. The layers areconstructed so that the first layer detects a set of primitive patternsin the input (e.g., image) data, the second layer detects patterns ofpatterns, and the third layer detects patterns of those patterns. Insome embodiments, a node in the neural network may have connections toall nodes in the immediately preceding layer and the immediate nextlayer. Thus, the layers may be referred to as fully connected layers. Insome embodiments, a node in the neural network may have connections toonly some of the nodes in the immediately preceding layer and theimmediate next layer. Thus, the layers may be referred to as sparselyconnected layers. Each neuron in the neural network consists of aweighted linear combination of its inputs and the computation on eachneural network layer may be described as a multiplication of an inputmatrix and a weight matrix. A bias matrix is then added to the resultingproduct matrix to account for the threshold of each neuron in the nextlevel. Further, an activation function is applied to each resultantvalue, and the resulting values are placed in the matrix for the nextlayer. Thus, the output from a node i in the neural network may berepresented as:

yi=f(Σxiwi+bi)

where f is the activation function, Σxiwi is the weighted sum of inputmatrix and bi is the bias matrix.

The activation function determines the activity level or excitationlevel generated in the node as a result of an input signal of aparticular size. The purpose of the activation function is to introducenon-linearity into the output of a neural network node because mostreal-world functions are non-linear and it is desirable that the neuronscan learn these non-linear representations. Several activation functionsmay be used in an artificial neural network. One example activationfunction is the sigmoid function σ(x), which is a continuous S-shapedmonotonically increasing function that asymptotically approaches fixedvalues as the input approaches plus or minus infinity. The sigmoidfunction σ(x) takes a real-valued input and transforms it into a valuebetween 0 and 1:

σ(x)=1/(1+exp(−x)).

Another example activation function is the tanh function, which takes areal-valued input and transforms it into a value within the range of[−1, 1]:

tanh(x)=2σ(2x)−1

A third example activation function is the rectified linear unit (ReLU)function. The ReLU function takes a real-valued input and thresholds itabove zero (i.e., replacing negative values with zero):

ƒ(x)=max(0,x).

It will be apparent that the above activation functions are provided asexamples and in various embodiments, neural network may utilize avariety of activation functions including (but not limited to) identity,binary step, logistic, soft step, tan h, arctan, softsign, rectifiedlinear unit (ReLU), leaky rectified linear unit, parameteric rectifiedlinear unit, randomized leaky rectified linear unit, exponential linearunit, s-shaped rectified linear activation unit, adaptive piecewiselinear, softplus, bent identity, softexponential, sinusoid, sinc,gaussian, softmax, maxout, and/or a combination of activation functions.

In embodiments, the input layer may take external inputs x1, x2, x3, x4and x5, which may be numerical values depending upon the input dataset.It will be understood that a node may include tens, hundreds, thousands,or more inputs. As discussed above, no computation is performed on theinput layer and thus the outputs are x1, x2, x3, x4 and x5 respectively,which are fed into a hidden layer. The output of nodes in the hiddenlayer may depend on the outputs from the input layer (x1, x2, x3, x4 andx5) and weights associated with connections (w1, w2, w3, w4 and w5)between the input layer and the hidden layer.

The outputs from the nodes and in the hidden layer may also be computedin a similar manner and then be fed to the node in the output layer.Node in the output layer may perform similar computations (using weightsv1, v2 and v3 associated with the connections) as the nodes, and in thehidden layers.

As mentioned, the connections between nodes in the neural network haveassociated weights, which determine how much relative effect an inputvalue has on the output value of the node in question. Before thenetwork is trained, random values are selected for each of the weights.The weights are adjusted during the training process and this adjustmentof weights to determine the best set of weights that maximize theaccuracy of the neural network is referred to as training. For everyinput in a training dataset, the output of the artificial neural networkmay be observed and compared with the expected output, and the errorbetween the expected output and the observed output may be propagatedback to the previous layer. The weights may be adjusted accordinglybased on the error. This process is repeated until the output error isbelow a predetermined threshold.

In embodiments, backpropagation (e.g., backward propagation of errors)is utilized with an optimization method such as gradient descent toadjust weights and update the neural network characteristics.Backpropagation may be a supervised training scheme that learns fromlabeled training data and errors at the nodes by changing parameters ofthe neural network to reduce the errors. For example, a result offorward propagation (e.g., output activation value(s)) determined usingtraining input data is compared against a corresponding known referenceoutput data to calculate a loss function gradient. The gradient may bethen utilized in an optimization method to determine new updated weightsin an attempt to minimize a loss function. For example, to measureerror, the mean square error is determined using the equation:

E=(target−output)²

To determine the gradient for a weight “w,” a partial derivative of theerror with respect to the weight may be determined, where:

gradient=∂E/∂w

The calculation of the partial derivative of the errors with respect tothe weights may flow backwards through the node levels of the neuralnetwork. Then a portion (e.g., ratio, percentage, etc.) of the gradientis subtracted from the weight to determine the updated weight. Theportion may be specified as a learning rate “a.” Thus an exampleequation of determining the updated weight is given by the formula:

w new=w old−α∂E/∂w

The learning rate must be selected such that it is not too small (e.g.,a rate that is too small may lead to a slow convergence to the desiredweights) and not too large (e.g., a rate that is too large may cause theweights to not converge to the desired weights).

After the weight adjustment, the network should perform better thanbefore for the same input because the weights have now been adjusted tominimize the errors.

As mentioned, neural networks may include convolutional neural networks(CNN). A CNN is a specialized neural network for processing data havinga known, grid-like topology, such as image data. Accordingly, CNNs arecommonly used for classification, object recognition and computer visionapplications, but they also may be used for other types of patternrecognition such as speech and language processing.

A convolutional neural network learns highly non-linear mappings byinterconnecting layers of artificial neurons arranged in many differentlayers with activation functions that make the layers dependent. Itincludes one or more convolutional layers, interspersed with one or moresub-sampling layers and non-linear layers, which are typically followedby one or more fully connected layers.

In embodiments, a CNN includes an input layer with an input image to beclassified by the CNN, a hidden layer, which in turn includes one ormore convolutional layers interspersed with one or more activation ornon-linear layers (e.g., ReLU), pooling or sub-sampling layers, and anoutput layer, which typically includes one or more fully connectedlayers. Input image may be represented by a matrix of pixels and mayhave multiple channels. For example, a colored image may have red,green, and blue channels each representing red, green, and blue (RGB)components of the input image. Each channel may be represented by a 2-Dmatrix of pixels having pixel values in the range of 0 to 255. Agray-scale image on the other hand may have only one channel. Thefollowing section describes processing of a single image channel usingCNN. It will be understood that multiple channels may be processed in asimilar manner.

As shown, input image may be processed by the hidden layer, whichincludes sets of convolutional and activation layers each followed bypooling layers.

The convolutional layers of the convolutional neural network serve asfeature extractors capable of learning and decomposing the input imageinto hierarchical features. The convolution layers may performconvolution operations on the input image where a filter (also referredto as a kernel or feature detector) may slide over the input image at acertain step size (referred to as the stride). For every position (orstep), element-wise multiplications between the filter matrix and theoverlapped matrix in the input image may be calculated and summed to geta final value that represents a single element of an output matrixconstituting a feature map. The feature map refers to image data thatrepresents various features of the input image data and may have smallerdimensions as compared to the input image. The activation or non-linearlayers use different non-linear trigger functions to signal distinctidentification of likely features on each hidden layer. Non-linearlayers use a variety of specific functions to implement the non-lineartriggering, including the rectified linear units (ReLUs), hyperbolictangent, absolute of hyperbolic tangent and sigmoid functions. In oneimplementation, a ReLU activation implements the function y=max(x, 0)and keeps the input and output sizes of a layer the same. The advantageof using ReLU is that the convolutional neural network is trained manytimes faster. ReLU is a non-continuous, non-saturating activationfunction that is linear with respect to the input if the input valuesare larger than zero and zero otherwise.

In one example, the first convolution and activation layer may performconvolutions on input image using multiple filters followed bynon-linearity operation (e.g., ReLU) to generate multiple outputmatrices (or feature maps). The number of filters used may be referredto as the depth of the convolution layer. Thus, the first convolutionand activation layer in the example has a depth of three and generatesthree feature maps using three filters. Feature maps may then be passedto the first pooling layer that may sub-sample or down-sample thefeature maps using a pooling function to generate output matrix. Thepooling function replaces the feature map with a summary statistic toreduce the spatial dimensions of the extracted feature map therebyreducing the number of parameters and computations in the network. Thus,the pooling layer reduces the dimensionality of the feature maps whileretaining the most important information. The pooling function can alsobe used to introduce translation invariance into the neural network,such that small translations to the input do not change the pooledoutputs. Different pooling functions may be used in the pooling layer,including max pooling, average pooling, and 12-norm pooling.

Output matrix may then be processed by a second convolution andactivation layer to perform convolutions and non-linear activationoperations (e.g., ReLU) as described above to generate feature maps. Inembodiments, a second convolution and activation layer may have a depthof five. Feature maps may then be passed to a pooling layer, wherefeature maps may be subsampled or down-sampled to generate an outputmatrix.

Output matrix generated by the pooling layer is then processed by one ormore fully connected layers that forms a part of the output layer ofCNN. The fully connected layer has a full connection with all thefeature maps of the output matrix of the pooling layer. In embodiments,the fully connected layer may take the output matrix generated by thepooling layer as the input in vector form and perform high-leveldetermination to output a feature vector containing information of thestructures in the input image. In embodiments, the fully connected layermay classify the object in input image into one of several categoriesusing a Softmax function. The Softmax function may be used as theactivation function in the output layer and takes a vector ofreal-valued scores and maps it to a vector of values between zero andone that sum to one. In embodiments, other classifiers, such as asupport vector machine (SVM) classifier, may be used.

In embodiments, one or more normalization layers may be added to the CNNto normalize the output of the convolution filters. The normalizationlayer may provide whitening or lateral inhibition, avoid vanishing orexploding gradients, stabilize training, and enable learning with higherrates and faster convergence. In embodiments, the normalization layersare added after the convolution layer but before the activation layer.

CNN may thus be seen as multiple sets of convolution, activation,pooling, normalization and fully connected layers stacked together tolearn, enhance and extract implicit features and patterns in the inputimage. A layer, as used herein, can refer to one or more components thatoperate with similar function by mathematical or other functional meansto process received inputs to generate/derive outputs for a next layerwith one or more other components for further processing within CNN.

The initial layers of CNN (e.g., convolution layers) may extract lowlevel features such as edges and/or gradients from the input image.Subsequent layers may extract or detect progressively more complexfeatures and patterns such as presence of curvatures and textures inimage data and so on. The output of each layer may serve as an input ofa succeeding layer in CNN to learn hierarchical feature representationsfrom data in the input image. This allows convolutional neural networksto efficiently learn increasingly complex and abstract visual concepts.

Although only two convolution layers are shown in the example, thepresent disclosure is not limited to the example architecture, and CNNarchitecture may comprise any number of layers in total, and any numberof layers for convolution, activation, and pooling. For example, therehave been many variations and improvements over the basic CNN modeldescribed above. Some examples include Alexnet, GoogLeNet, VGGNet (thatstacks many layers containing narrow convolutional layers followed bymax pooling layers), Residual network or ResNet (that uses residualblocks and skip connections to learn residual mapping), DenseNet (thatconnects each layer of CNN to every other layer in a feed-forwardfashion), Squeeze and excitation networks (that incorporate globalcontext into features) and AmobeaNet (that uses evolutionary algorithmsto search and find optimal architecture for image recognition).

The training process of a convolutional neural network, such as CNN, maybe similar to the training process discussed in FIG. 148 with respect toneural network.

In embodiments, all parameters and weights (including the weights in thefilters and weights for the fully-connected layer are initially assigned(e.g., randomly assigned). Then, during training, a training image orimages, in which the objects have been detected and classified, areprovided as the input to the CNN, which performs the forward propagationsteps. In other words, CNN applies convolution, non-linear activation,and pooling layers to each training image to determine theclassification vectors (i.e., detect and classify each training image).These classification vectors are compared with the predeterminedclassification vectors. The error (e.g., the squared sum of differences,log loss, Softmax log loss) between the classification vectors of theCNN and the predetermined classification vectors is determined. Thiserror is then employed to update the weights and parameters of the CNNin a backpropagation process which may use gradient descent and mayinclude one or more iterations. The training process is repeated foreach training image in the training set.

The training process and inference process described above may beperformed on hardware, software, or a combination of hardware andsoftware. However, training a convolutional neural network like CNN orusing the trained CNN for inference generally requires significantamounts of computation power to perform, for example, the matrixmultiplications or convolutions. Thus, specialized hardware circuits,such as graphic processing units (GPUs), tensor processing units (TPUs),neural network processing units (NPUs), FPGAs, ASICs, or other highlyparallel processing circuits may be used for training and/or inference.Training and inference may be performed on a cloud, on a data center, oron a device.

In embodiments, an object detection model extends the functionality ofCNN-based image classification neural network models by not onlyclassifying objects but also determining their locations in an image interms of bounding boxes. Region-based CNN (R-CNN) methods are used toextract regions of interest (ROI), where each ROI is a rectangle thatmay represent the boundary of an object in image. Conceptually, R-CNNoperates in two phases. In a first phase, region proposal methodsgenerate all potential bounding box candidates in the image. In a secondphase, for every proposal, a CNN classifier is applied to distinguishbetween objects. Alternatively, a fast R-CNN architecture can be used,which integrates the feature extractor and classifier into a unifiednetwork. Another faster R-CNN can be used, which incorporates a RegionProposal Network (RPN) and fast R-CNN into an end-to-end trainableframework. Mask R-CNN adds instance segmentation, while mesh R-CNN addsthe ability to generate a 3D mesh from a 2D image.

In embodiments, artificial intelligence modules 13404 may provide accessto and/or integrate a robotic process automation (RPA) module 13416. TheRPA module 13416 may facilitate, among other things, computer automationof producing and validating workflows. In embodiments, an RPA module13416 may monitor human interaction with various systems to learnpatterns and processes performed by humans in performance of respectivetasks. This may include observation of human actions that involveinteractions with hardware elements, with software interfaces, and withother elements. Observations may include field observations as humansperform real tasks, as well as observations of simulations or otheractivities in which a human performs an action with the explicit intentto provide a training data set or input for the RPA system, such aswhere a human tags or labels a training data set with features thatassist the RPA system in learning to recognize or classify features orobjects, among many other examples. In embodiments, an RPA module 13416may learn to perform certain tasks based on the learned patterns andprocesses, such that the tasks may be performed by the RPA module 13416in lieu or in support of a human decision maker. Examples of RPA modules13416 may encompass those in this disclosure and in the documentsincorporated by reference herein and may involve automation of any ofthe wide range of value chain network activities or entities describedtherein.

In embodiments, the artificial intelligence modules 13404 may includeand/or provide access to an analytics module 13418. In embodiments, ananalytics module 13418 is configured to perform various analyticalprocesses on data output from value chain entities or other datasources. In example embodiments, analytics produced by the analyticsmodule 13418 may facilitate quantification of system performance ascompared to a set of goals and/or metrics. The goals and/or metrics maybe preconfigured, determined dynamically from operating results, and thelike. Examples of analytics processes that can be performed by ananalytics module 13418 are discussed below and in the documentincorporated herein by reference. In some example implementations,analytics processes may include tracking goals and/or specific metricsthat involve coordination of value chain activities and demandintelligence, such as involving forecasting demand for a set of smartcontainers by location and time (among many others).

In embodiments, artificial intelligence modules 13404 may include and/orprovide access to a digital twin module 13420. The digital twin module13420 may encompass any of a wide range of features and capabilitiesdescribed herein. In embodiments, a digital twin module 13420 may beconfigured to provide, among other things, execution environments fordifferent types of digital twins, such as smart container digital twins13504, digital twins of physical shipping environments (shipping yard,container port, or the like), digital twins of modes of transportation(container ship, truck, railway, or the like), digital twins of smartcontainer operating units, logistics digital twins, organizationaldigital twins, role-based digital twins, and the like. In embodiments,the digital twin module 13420 may be configured in accordance withdigital twin systems and/or modules described elsewhere throughout thedisclosure. In example embodiments, a digital twin module 13420 may beconfigured to generate digital twins that are requested by intelligenceservice clients. Further, the digital twin module 13420 may beconfigured with interfaces, such as APIs and the like for receivinginformation from external data sources. For instance, the digital twinmodule 13420 may receive real-time data from sensor systems of a smartcontainer, machinery, vehicle, robot, or other device, and/or sensorsystems of the physical environment in which a device operates. Inembodiments, the digital twin module 13420 may receive digital twin datafrom other suitable data sources, such as 3^(rd) party services (e.g.,weather services, traffic data services, logistics systems anddatabases, and the like. In embodiments, the digital twin module 13420may include digital twin data representing features, states, or the likeof value chain network entities, such as supply chain infrastructureentities, transportation or logistic entities, containers, goods, or thelike, as well as demand entities, such as customers, merchants, stores,points-of-sale, points-of-use, and the like. The digital twin module13420 may be integrated with or into, link to, or otherwise interactwith an interface (e.g., a control tower or dashboard), for coordinationof supply and demand, including coordination of automation within supplychain activities and demand management activities.

In embodiments, a digital twin module 13420 may provide access to andmanage a library of digital twins. Artificial intelligence modules8 mayaccess the library to perform functions, such as a simulation of actionsin a given environment in response to certain stimuli.

In embodiments, artificial intelligence modules 13404 may include and/orprovide access to a machine vision module 13422. In embodiments, amachine vision module 13422 is configured to process images (e.g.,captured by a camera, a liquid lens system, or the like) to detect andclassify objects in the image. In embodiments, the machine vision module13422 receives one or more images (which may be frames of a video feedor single still shot images) and identifies “blobs” in an image (e.g.,using edge detection techniques or the like). The machine vision module13422 may then classify the blobs. In some embodiments, the machinevision module 13422 leverages one or more machine-learned imageclassification models and/or neural networks (e.g., convolutional neuralnetworks) to classify the blobs in the image. In some embodiments, themachine vision module 13422 may perform feature extraction on the imagesand/or the respective blobs in the image prior to classification. Insome embodiments, the machine vision module 13422 may leverageclassification made in a previous image to affirm or updateclassification(s) from the previous image. For example, if an objectthat was detected in a previous frame was classified with a lowerconfidence score (e.g., the object was partially occluded or out offocus), the machine vision module 13422 may affirm or update theclassification if the machine vision module 13422 is able to determine aclassification of the object with a higher degree of confidence. Inembodiments, the machine vision module 13422 is configured to detectocclusions, such as objects that may be occluded by another object. Inembodiments, the machine vision module 13422 receives additional inputto assist in image classification tasks, such as from a radar, a sonar,a digital twin of an environment (which may show locations of knownobjects), and/or the like. In some embodiments, a machine-vision module13322 may include or interface with a liquid lens. In these embodiments,the liquid lens may facilitate improved machine vision (e.g., whenfocusing at multiple distances is necessitated by the environment of asmart container and/or within the smart container) and/or other machinevision tasks that are enabled by a liquid lens.

In embodiments, the artificial intelligence modules 13404 may includeand/or provide access to a natural language processing (NLP) module13424. In embodiments, an NLP module 13424 performs natural languagetasks on behalf of an intelligence service client 13324. Examples ofnatural language processing techniques may include, but are not limitedto, speech recognition, speech segmentation, speaker diarization,text-to-speech, lemmatization, morphological segmentation,parts-of-speech tagging, stemming, syntactic analysis, lexical analysis,and the like. In embodiments, the NLP module 13424 may enable voicecommands that are received from a human. In embodiments, the NLP module13424 receives an audio stream (e.g., from a microphone) and may performvoice-to-text conversion on the audio stream to obtain a transcriptionof the audio stream. The NLP module 13424 may process text (e.g., atranscription of the audio stream) to determine a meaning of the textusing various NLP techniques (e.g., NLP models, neural networks, and/orthe like). In embodiments, the NLP module 13424 may determine an actionor command that was spoken in the audio stream based on the results ofthe NLP. In embodiments, the NLP module 13424 may output the results ofthe NLP to an intelligence service client 13324.

In embodiments, the NLP module 13424 provides an intelligence serviceclient 13324 with the ability to parse one or more conversational voiceinstructions provided by a human user to perform one or more tasks aswell as communicate with the human user. The NLP module 13424 mayperform speech recognition to recognize the voice instructions, naturallanguage understanding to parse and derive meaning from theinstructions, and natural language generation to generate a voiceresponse for the user upon processing of the user instructions. In someembodiments, the NLP module 13424 enables an intelligence service client13324 to understand the instructions and, upon successful completion ofthe task by the intelligence service client 13324, provide a response tothe user. In embodiments, the NLP module 13424 may formulate and askquestions to a user if the context of the user request is not completelyclear. In embodiments, the NLP module 13424 may utilize inputs receivedfrom one or more sensors including vision sensors, location-based data(e.g., GPS data) to determine context information associated withprocessed speech or text data.

In embodiments, the NLP module 13424 uses neural networks whenperforming NLP tasks, such as recurrent neural networks, long short-termmemory (LSTMs), gated recurrent unit (GRUs), transformer neuralnetworks, convolutional neural networks and/or the like.

In an example neural network for implementing NLP module 13424, theneural network is a transformer neural network. In the example, thetransformer neural network includes three input stages and five outputstages to transform an input sequence into an output sequence. Theexample transformer includes an encoder and a decoder. The encoderprocesses input, and the decoder generates output probabilities, forexample. The encoder includes three stages, and the decoder includesfive stages. Encoder stage 1 represents an input as a sequence ofpositional encodings added to embedded inputs. Encoder stages 2 and 3include N layers (e.g., N=6, etc.) in which each layer includes aposition-wise feedforward neural network (FNN) and an attention-basedsublayer. Each attention-based sublayer of encoder stage 2 includes fourlinear projections and multi-head attention logic to be added andnormalized to be provided to the position-wise FNN of encoder stage 3.Encoder stages 2 and 3 employ a residual connection followed by anormalization layer at their output.

The example decoder processes an output embedding as its input with theoutput embedding shifted right by one position to help ensure that aprediction for position i is dependent on positions previous to/lessthan i. In stage 2 of the decoder, masked multi-head attention ismodified to prevent positions from attending to subsequent positions.Stages 3-4 of the decoder include N layers (e.g., N=6, etc.) in whicheach layer includes a position-wise FNN and two attention-basedsublayers. Each attention-based sublayer of decoder stage 3 includesfour linear projections and multi-head attention logic to be added andnormalized to be provided to the position-wise FNN of decoder stage 4.Decoder stages 2-4 employ a residual connection followed by anormalization layer at their output. Decoder stage 5 provides a lineartransformation followed by a Softmax function to normalize a resultingvector of K numbers into a probability distribution including Kprobabilities proportional to exponentials of the K input numbers.

In embodiments, artificial intelligence modules 13404 may also includeand/or provide access to a rules-based module 13428 that may beintegrated into or be accessed by an intelligence service client 13324.In some embodiments, a rules-based module 13428 may be configured withprogrammatic logic that defines a set of rules and other conditions thattrigger certain actions that may be performed in connection with anintelligence client. In embodiments, the rules-based module 13428 may beconfigured with programmatic logic that receives input and determineswhether one or more rules are met based on the input. If a condition ismet, the rules-based module 13428 determines an action to perform, whichmay be output to a requesting intelligence service client 13324. Thedata received by the rules-based engine may be received from anintelligence service inputs 13470 and/or may be requested from anothermodule in artificial intelligence modules 13404, such as the machinevision module 13422, the neural network module 13414, the machinelearning module 13412, and/or the like. For example, a rules-basedmodule 13428 may receive classifications of objects in a field of viewof a smart container from a machine vision system and/or sensor datafrom a lidar sensor of the smart container and, in response, maydetermine whether the smart container should continue in its path,change its course, or stop. In embodiments, the rules-based module 13428may be configured to make other suitable rules-based decisions on behalfof a respective intelligence service client 13324, examples of which arediscussed throughout the disclosure. In some embodiments, therules-based engine may apply governance standards and/or analysismodules, which are described in greater detail below.

In embodiments, artificial intelligence modules 13404 interface with anintelligence service controller 13402, which is configured to determinea type of request issued by an intelligence service client 13324 and, inresponse, may determine a set of governance standards and/or analysesthat are to be applied by the artificial intelligence modules 13404 whenresponding to the request. In embodiments, the intelligence servicecontroller 13402 may include an analysis management module 13406, a setof analysis modules 13408, and a governance library 13410.

In embodiments, an intelligence service controller 13402 is configuredto determine a type of request issued by an intelligence service client13324 and, in response, may determine a set of governance standardsand/or analyses that are to be applied by the artificial intelligencemodules 13404 when responding to the request. In embodiments, theintelligence service controller 13402 may include an analysis managementmodule 13406, a set of analysis modules 13408, and a governance library13410. In embodiments, the analysis management module 13406 receives anartificial intelligence module 13404 request and determines thegovernance standards and/or analyses implicated by the request. Inembodiments, the analysis management module 13406 may determine thegovernance standards that apply to the request based on the type ofdecision that was requested and/or whether certain analyses are to beperformed with respect to the requested decision. For example, a requestfor a control decision that results in an intelligence service client13324 performing an action may implicate a certain set of governancestandards that apply, such as safety standards, legal standards, qualitystandards, or the like, and/or may implicate one or more analysesregarding the control decision, such as a risk analysis, a safetyanalysis, an engineering analysis, or the like.

In some embodiments, the analysis management module 13406 may determinethe governance standards that apply to a decision request based on oneor more conditions. Non-limiting examples of such conditions may includethe type of decision that is requested, a geolocation in which adecision is being made, an environment that the decision will affect,current or predicted environment conditions of the environment and/orthe like. In embodiments, the governance standards may be defined as aset of standards libraries stored in a governance library 13410. Inembodiments, standards libraries may define conditions, thresholds,rules, recommendations, or other suitable parameters by which a decisionmay be analyzed. Examples of standards libraries may include a legalstandards library, a regulatory standards library, a quality standardslibrary, an engineering standards library, a safety standards library, afinancial standards library, and/or other suitable types of standardslibraries. In embodiments, the governance library 13410 may include anindex that indexes certain standards defined in the respective standardslibrary based on different conditions. Examples of conditions may be ajurisdiction or geographic areas to which certain standards apply,environmental conditions to which certain standards apply, device typesto which certain standards apply, materials or products to which certainstandards apply, and/or the like.

In some embodiments, the analysis management module 13406 may determinethe appropriate set of standards that must be applied with respect to aparticular decision and may provide the appropriate set of standards tothe artificial intelligence modules 13404, such that the artificialintelligence modules 13404 leverages the implicated governance standardswhen determining a decision. In these embodiments, the artificialintelligence modules 13404 may be configured to apply the standards inthe decision-making process, such that a decision output by theartificial intelligence modules 13404 is consistent with the implicatedgovernance standards. It is appreciated that the standards libraries inthe governance library may be defined by the platform provider,customers, and/or third parties. The standards may be governmentstandards, industry standards, customer standards, or other suitablesources. In embodiments, each set of standards may include a set ofconditions that implicate the respective set of standards, such that theconditions may be used to determine which standards to apply given asituation.

In some embodiments, the analysis management module 13406 may determineone or more analyses that are to be performed with respect to aparticular decision and may provide corresponding analysis modules 13408that perform those analyses to the artificial intelligence modules13404, such that the artificial intelligence modules 13404 leverage thecorresponding analysis modules 13408 to analyze a decision beforeoutputting the decision to the requesting client. In embodiments, theanalysis modules 13408 may include modules that are configured toperform specific analyses with respect to certain types of decisions,whereby the respective modules are executed by a processing system thathosts the instance of the intelligence service 13004. Non-limitingexamples of analysis modules 13408 may include risk analysis module(s),security analysis module(s), decision tree analysis module(s), ethicsanalysis module(s), failure mode and effects (FMEA) analysis module(s),hazard analysis module(s), quality analysis module(s), safety analysismodule(s), regulatory analysis module(s), legal analysis module(s),and/or other suitable analysis modules.

In some embodiments, the analysis management module 13406 is configuredto determine which types of analyses to perform based on the type ofdecision that was requested by an intelligence service client 13324. Insome of these embodiments, the analysis management module 13406 mayinclude an index or other suitable mechanism that identifies a set ofanalysis modules 13408 based on a requested decision type. In theseembodiments, the analysis management module 13406 may receive thedecision type and may determine a set of analysis modules 13408 that areto be executed based on the decision type. Additionally, oralternatively, one or more governance standards may define when aparticular analysis is to be performed. For example, the engineeringstandards may define what scenarios necessitate a FMEA analysis. In thisexample, the engineering standards may have been implicated by a requestfor a particular type of decision and the engineering standards maydefine scenarios when an FMEA analysis is to be performed. In thisexample, artificial intelligence modules 13404 may execute a safetyanalysis module and/or a risk analysis module and may determine analternative decision if the action would violate a legal standard or asafety standard. In response to analyzing a proposed decision,artificial intelligence modules 13404 may selectively output theproposed condition based on the results of the executed analyses. If adecision is allowed, artificial intelligence modules 13404 may outputthe decision to the requesting intelligence service client 13324. If theproposed configuration is flagged by one or more of the analyses,artificial intelligence modules 13404 may determine an alternativedecision and execute the analyses with respect to the alternate proposeddecision until a conforming decision is obtained.

It is noted here that in some embodiments, one or more analysis modules13408 may themselves be defined in a standard, and one or more relevantstandards used together may comprise a particular analysis. For example,the applicable safety standard may call for a risk analysis that can useone or more allowable methods. In this example, an ISO standard foroverall process and documentation, and an ASTM standard for a narrowlydefined procedure may be employed to complete the risk analysis requiredby the safety governance standard.

As mentioned, the foregoing framework of an intelligence service 13004may be applied and/or leveraged at various levels of a value chain. Forexample, in some embodiments, a platform level intelligence system maybe configured with the entire capabilities of the intelligence service13004, and certain configurations of the intelligence service 13004 maybe provisioned for respective value chain entities. Furthermore, in someembodiments, an intelligence service client 13324, such as the smartcontainer system 13000, may be configured to escalate an intelligencesystem task to a higher-level value chain entity (e.g., edge-level orthe platform-level) when the intelligence service client 13324 cannotperform the task autonomously. It is noted that in some embodiments, anintelligence service controller 13402 may direct intelligence tasks to alower-level component. Furthermore, in some implementations, anintelligence service 13004 may be configured to output default actionswhen a decision cannot be reached by the intelligence service 13004and/or a higher or lower-level intelligence system. In some of theseimplementations, the default decisions may be defined in a rule and/orin a standards library.

In embodiments, a “set” of machine-learned models may include a set withmultiple members. In embodiments, a “set” of machine-learned models mayinclude hybrids of different types of models (e.g., hybrids of RNN andCNN).

In one example, a set of machine-learned models may be used for smartcontainer predictive maintenance. In this example, the intelligenceservice 13004 may receive order data, historical order data, maintenancedata, weather data, and/or video feed from sensors inside a smartcontainer for the user device 13094 and may generate a set of featurevectors based on the received data. The intelligence service 13004 mayinput the feature vectors into machine-learned models trained (e.g.,using a combination of simulation data and real-world data) to predictwhen a particular smart container will require maintenance, such asbased on a training data set of outcomes. In embodiments, theintelligence service 13004 may include an input set of training datarepresenting predictions or the probability of required maintenance by aset of human experts and/or by other systems or models.

In yet another example, a set of machine-learned models may be used topredict the traffic at a container terminal at a given point in time. Inthis example, the intelligence service 13004 may receive historicalcontainer terminal traffic data, maritime data, news data, and weatherdata and may generate feature vectors based on the received data. Inembodiments, feature vectors may include other data, such as datacharacterizing container terminal layout elements upon which traffic maydepend. The intelligence service 13004 may input the feature vectorsinto machine-learned models trained (e.g., using a combinationsimulation data and real-world data) to predict the traffic at acontainer terminal.

In another example, a set of machine-learned models may be used todetect illicit and/or illegal items being shipped. In this example, theintelligence service 13004 may receive order data, shipper data,historical cargo data, and/or video feed and other sensor data fromsensors disposed inside of a smart container and may generate featurevectors based on the received data. The intelligence service 13004 mayinput the feature vectors into machine-learned models trained (e.g.,using a combination of simulation data and real-world data) to detectillicit and/or illegal items. In embodiments, detection of illicitand/or illegal items may involve a set of distinct models that arerespectively trained based on training data sets and/or feature vectorinputs that are specific to jurisdictional factors, including laws orregulations (e.g., training with awareness of legality), culturalfactors (e.g., where whether the item is considered illicit varies basedon cultural norms, and the like. In embodiments, training may includeproviding, such as through human experts, information about alternativeterminology, or the like, that shippers or other users may employ todescribe illegal or illicit items (such as when shippers or other usersdescribe the cargo to be shipped in an order), such as code words,euphemisms, or the like. In embodiments, a model may be trained toprovide a word cloud, cluster of words, or other features, such as tofacilitate recognition of illegal or illicit items and/or recognition ofwords, images, or other elements used to characterize them. As onenon-limiting example, a self-organizing map (SOM) may be employed togenerate a mapping of entities, such as mapping entities, classes,objects, workflows, or the like to jurisdictions, to topics, to eachother, or the like. Additionally, or alternatively, the machine-learnedmodels may be configured to identify container contents.

In another example, a set of machine-learned models may be used toprovide decision support related to the pricing of one or more freightstorage and/or transportation services (e.g., services that require theuse of a smart container). For example, the intelligence service 13004may receive data from various sources described throughout this documentand the documents incorporated by reference herein and may generate aset of feature vectors based on the received data. The intelligenceservice 13004 may input the feature vectors into machine-learned modelstrained (e.g., using a combination of simulation data and real-worlddata) to provide decision support related to the pricing of one or moreservices, such as based on a training data set of outcomes. Inembodiments, the intelligence service 13004 may include an input set oftraining data representing decision support related to service pricingby a set of human experts and/or by other systems or models. Datasources used to produce the set of feature vectors, may include, but arenot limited to, order data, demand data, supply data, cost data,volatility data, pricing pattern data, order size data, order volumedata, geographic trading data, maritime data, trucking fleet data,railway data, traffic data, weather data, social media sites, externaldata (such as news involving smart containers or shipping or the like),and many others.

In another example, a set of machine-learned models may be used toprovide decision support related to loading and/or unloading cargo. Theintelligence service 13004 may receive order data (optionally includingweight data, volume data, cargo description data, destination location,or the like) and/or video feed from sensors disposed outside the smartcontainer and/or within the smart container and may generate a set offeature vectors based on the received data. The intelligence service13004 may input the set of feature vectors into a machine-learned modeltrained (e.g., using a combination of simulation data and real-worlddata) to provide decision support related to cargo loading and/orunloading. For example, the machine-learned model could be configured toprovide decision support about the order in which specific cargo isloaded and/or unloaded, the clustering of cargo, the configuration ofcargo within the smart container, or the like. In embodiments, a modelor set of models may be trained by an expert in the loading and/orunloading of cargo.

In yet another example, a set of machine-learned models may be used todetermine regulatory compliance of a shipment. For example, theintelligence service 13004 may receive data from various sourcesdescribed throughout this document and the documents incorporated byreference herein and may generate a set of feature vectors based on thereceived data. The intelligence service 13004 may input the featurevectors into machine-learned models trained to determine regulatorycompliance. As one non-limiting example, regulatory compliance mayinclude compliance with regulations that require documentationconfirming customs duties are paid. In embodiments, relating to such anexample, a machine-learned model may parse documentation, commercialinvoices, and the like, such as to find verification of the requiredtariff payments.

In another example, a set of machine-learned models may be used tocategorize or classify cargo. For example, the intelligence service13004 may receive data from various sources described throughout thisdocument and the documents incorporated by reference herein and maygenerate a set of feature vectors based on the received data. Theintelligence service 13004 may input the set of feature vectors into amachine-learned model trained (e.g., using a combination of simulationdata and real-world data) to categorize cargo, such as based on atraining data set of outcomes. In embodiments, the intelligence service13004 may include an input set of training data representingcategorizations or classifications of cargo by a set of human expertsand/or by other systems or models. Data sources and feature vectors usedfor categorization or classification of cargo may include shipping dataof the many types described herein, shipper profile data, as well asexternal data sources that may assist with classification orcategorization of cargo. Such artificial intelligence systems used forclassification, in the present example and other examples describedherein, may include a recurrent neural network (including a gatedrecurrent neural network), a convolutional neural network, a combinationof a recurrent neural network and a convolutional neural network, orother type of neural network or combination or hybrid of types of neuralnetwork described herein or in the documents incorporated by referenceherein.

In another example, a set of machine-learned models may be used tooptimize smart container design. For example, the intelligence service13004 may receive data from various sources described throughout thisdocument and the documents incorporated by reference herein and maygenerate a set of feature vectors based on the received data. Theintelligence service 13004 may input the feature vectors intomachine-learned models trained (e.g., using a combination of simulationdata and real-world data) to optimize the design of the smart container,such as based on a training data set of outcomes. In embodiments, theintelligence service 13004 may include an input set of training datarepresenting smart container optimization by a set of human expertsand/or by other systems or models. Data sources and feature vectors usedfor optimization of marketplace efficiency may include shipping data ofthe many types described herein that may assist with smart containerdesign optimization. Such artificial intelligence systems used foroptimization, in the present example and other examples describedherein, may include a recurrent neural network (including a gatedrecurrent neural network), a convolutional neural network, a combinationof a recurrent neural network and a convolutional neural network, orother type of neural network or combination or hybrid of types of neuralnetwork described herein or in the documents incorporated by referenceherein. In embodiments, the smart container design may be optimized forcost, carbon emissions, speed, efficiency, performance, performance inspecific environments (e.g., optimized to operate in arctic conditions),carrying capacity, safety, and the like.

In another example, a set of machine-learned models may be used tooptimize a smart container route. For example, the intelligence service13004 may receive data from various sources described throughout thisdocument and the documents incorporated by reference herein and maygenerate a set of feature vectors based on the received data. Theintelligence service 13004 may input the feature vectors intomachine-learned models trained (e.g., using a combination of simulationdata and real-world data) to optimize the smart container route, such asbased on a training data set of outcomes. In embodiments, theintelligence service 13004 may include an input set of training datarepresenting marketplace profitability optimization by a set of humanexperts and/or by other systems or models. Data sources and featurevectors used for optimization of the smart container route may includeshipping data of the many types described herein that may assist withthe smart container route optimization, including historical route data,order data, weather data, maritime data, traffic data, truck fleet data,railway data, news data, or the like.

The foregoing examples are non-limiting examples and the intelligenceservice 13004 may be used for any other suitable AI/machine-learningrelated tasks that are performed with respect to smart containers andshipping environments.

In embodiments, a security system 13046 includes a framework that may beimplemented at various levels of the disclosed systems. In theseembodiments, instances of the security system 13046 may be implementedat the system-level, at the fleet- or team-level, or individual-level.For example, at the system-level, the security system 13046 may providesecurity-related functionality on behalf of the system 13000 and/or withrespect to any communications and/or other interactions with smartcontainer operating units. In embodiments, a security system 13046implemented at the fleet-level or team-level, whereby thesecurity-system may be configured to provide security-relatedfunctionality on behalf of the smart container team or fleet and/or withrespect to communications and/or other interactions with smartcontainers in the team or fleet. In embodiments, a security system 13046implemented at the smart container-level may be configured to providesecurity-related functionality on behalf of the smart container and/orwith respect to communications and/or other interactions with othersmart containers, smart container teams, and/or the system 13000.

In embodiments, a security system 13046 may include an autonomousadaptive security module, an autonomous non-adaptive security module,and/or a manual security module. An autonomous adaptive security modulemay be configured to request intelligence tasks from intelligenceservice 13004, whereby an adaptive security module leverages theartificial intelligence modules of an intelligence service 13004 toassess a security risk and determine an action based on an output of theintelligence service 13004. For example, the adaptive security module ofa smart container fleet may monitor one or more conditions associatedwith the smart container fleet by receiving data from a set of datasources, such as monitoring a route for potentially dangerous conditionsbased on a set of data sources (e.g., weather data, video feeds, sensordata from the smart containers and/or environment, input from individualsmart containers, and/or the like). In response to receiving the data,the adaptive security module may request an assessment (e.g., aclassification) of an environment from the intelligence service 13004regarding the security of the environment. In response, the intelligenceservice 13004 may provide one or more classifications that indicate anassessment of the environment. The adaptive security module may thendetermine whether the assessment necessitates an action to be taken, andif so, what particular action to take. In some of these embodiments, theadaptive security module may use a rules-based approach to determinewhether the assessment necessitates an action and, if so, what action totake. Additionally, or alternatively, the adaptive security module mayleverage a neural network that is trained to an action to recommendgiven a set of features (e.g., classifications, sensor readings from oneor more smart containers, locations of smart containers, objectsdetected in the environment and locations thereof, and/or any otherrelevant features). In these embodiments, the neural network module13414 may receive the features from the adaptive security module and/ora set of intelligence service inputs 13470 and may output a proposedaction given the set of features. In some of these embodiments, anintelligence service controller 13402 of the intelligence service 13004may allow or override decisions made by the artificial intelligencemodules 13404. For instance, the analysis modules 13408 may performdynamic risk analyses and/or static risk analyses. Examples of dynamicrisk analysis may include, but are not limited to, real-time data drivenanalyses (e.g., current weather patterns, current regulatoryenvironment, current container port events, and/or the like) and/or riskanalyses specific to a particular freight storage and/or transportationservice order (e.g., contractual risks, environmental risks, safetyliabilities, monetary liabilities, and/or the like). Examples of staticrisk analyses may include, but are not limited to, operational risksand/or regulatory/compliance risks.

In embodiments, the autonomous adaptive security module may operate inan isolated manner (e.g., without communication with external devices orsystems) or in a connected manner (e.g., with communication withexternal devices or systems).

In embodiments, the security system 13046 may include an autonomousnon-adaptive security module. In embodiments, the autonomousnon-adaptive security module is configured to make security relateddecisions on behalf of a client autonomously (e.g., without humanintervention). In embodiments, a non-adaptive security module performslogic-based security-related actions (e.g., risk mitigation actions) inresponse to detecting one or more specific sets of conditions. Forexample, a non-adaptive security module may be configured to, inresponse to detecting a specific set of conditions, trigger actions,such actions as, locking a smart container, locking the digital twin ofa smart container, shutting off smart container power, stopping amovement of the smart container, initiating charging, sounding an alarmor siren, triggering a strobe or light, sending a notification toanother device or system, self-destructing, or the like. In embodiments,the non-adaptive security module responds to risks that are more easilydiagnosable, such as overheating conditions, moving or being taken outof a geofenced area, detected internal leaks, low power conditions, lowfluid levels, and/or the like.

In embodiments, the security system 13046 may include a manual securitymodule. In embodiments, the manual security module is configured toallow a user to make decisions regarding security-related actions. Insome of these embodiments, the manual security module is configured toreceive a notification of an assessed risk (e.g., from the adaptivesecurity module, the non-adaptive security module, from an intelligenceservice client 13324, or the like). In these embodiments, the human usermay interface with the manual security module via a human interface,which may be provided via a user device (e.g., mobile device, tablet,computing device, or the like).

In embodiments, the fleet management system 13002 may utilize thefeatures and capabilities of the smart container system 13000 tofacilitate substantially optimized utilization of fleet resources byanticipating fleet resource needs and preparing those resources inadvance of anticipated use. In embodiments, resource need anticipationmay include coordinating maintenance activities with shipment schedulingto ensure that preventable interruptions due to lack of maintenance areprevented. Additionally, or alternatively, resource need anticipationmay be based on alignment of detected fleet resource use withinformation that supports, among other things, anticipation of freightstorage and/or transportation service orders. In embodiments, factorssuch as weather pattern forecasting, time of year, location, and/or thelike may influence the likelihood of certain freight storage and/ortransportation service orders. For example, high freight volumes arelikely during peak shipping season, from August to October, whereasfreight volumes are likely to be low during the start of the year, fromJanuary to March. Example implementations for generating fleet needpredictions and addressing those predictions follow the discussion ofthe components of the fleet management system 13002 and those of therelated smart container system 13000. As previously discussed, examplecomponents of the smart container system 13000 may include acommunication management system 13010, the remote-control system 13012,and a human interface system 13038.

In embodiments, the communication management system 13010 is configuredto enable communication (e.g., efficient and/or high speedcommunication) among system elements, such as the fleet managementsystem 13002 and its elements as described herein, the intelligenceservice 13004 and its elements as described herein, external datasources 13036, third party systems (e.g., via an Internet and the like),smart container operating units, support systems and equipment,transportation resources (e.g., container ships, trucks, rail, or thelike), human fleet resources, and the like. The communication managementsystem 13010 may include or provide access to one or more communicationnetwork types, such as wired, wireless and the like that may supportvarious data protocols, such as Internet Protocol (IP) and the like. Thecommunication management system may include or have access tointelligence services (e.g., via the fleet intelligence system resourcesdescribed herein) that manage and control portions of the smartcontainer fleet management system infrastructure associated withcommunication to ensure, for example: timely delivery of data collectedby deployed smart container operating units to critical computation,analysis and/or data storage resources; prioritized delivery of smartcontainer configuration and operational instructions; and the like. Infleet resource management and control embodiments, the communicationmanagement system 13010 may prioritize fleet security systemcommunications use of fleet communication resources over communicationsamong fleet intelligence system components to support a high degree ofsecurity and integrity of fleet resources. The communication managementsystem 13010 may provide and manage access to networking, includingnetwork system 13202 that connects at least the smart container system13000 with external systems, deployed smart container operating units,and other network-connectable elements (e.g., fleet edge devices and thelike).

In embodiments, capabilities of the communication management system13010 may include contextual specification, and/or adaptation of smartcontainer system communication resources (e.g., networks, radio systems,data communication devices, such as routers, and the like) based on,among other things, an order execution plan, plan definitions, taskdefinitions, smart container operating unit configurations, real-timejob status, and the like. The communication management system 13010adaptation of fleet communication resources may be impacted by a rangeof real-world conditions (e.g., weather, atmospheric conditions,container port traffic, container port and other facility structures,environment (e.g., land-to-submerged, subterranean), and the like). Inembodiments, the communication management system 13010 may glean contextfrom a freight storage and/or transportation service order that mayfacilitate anticipating a need for types of adaptation during orderexecution. As an example of freight storage and/or transportationservice order context-based communication adaptation, a job may initiateat sea level, and then include actions by subterranean teams.Communication resources suitable for use in these different taskenvironments that are configured by the fleet configuration systemduring job configuration activities may be adaptively controlled by thecommunication management system 13010 for the respective teams of smartcontainers as a job progresses through the exemplary environments.

Freight storage and/or transportation service order criteria maydirectly call for isolated operation. Alternatively, circumstances ofthe freight storage and/or transportation service order may favorisolated operation (e.g., operation within a foreign jurisdiction andthe like). Communication resources for the requested service may beadapted accordingly. As an example, communication among a team of fleetresources assigned to co-locate when performing a job (e.g., co-locatingto an origin location) may be configured by the fleet configurationsystem with additional encryption or with a radio frequency that defiesconventional detection that the communication management system mayfacilitate activating when required by the freight storage and/ortransportation service order (e.g., as noted above when the team entersa foreign jurisdiction). In this further embodiment of fleet resourceconfiguration, the communication management system 13010 may detect andcontrol communication resources (e.g., smart container operating unitradio interfaces, communication infrastructure that is proximal toisolated smart container operating units and the like) to enforce such afleet configuration. Yet further consideration for isolated operationmay include adaptable isolation communication protocols, such aspermitting only use of low energy near-field communication conditionallybased on deployment context, such as an expected location of team smartcontainers, such as when multiple smart container operating units areexpected to be nearby. The communication management system 13010 mayassist the fleet configuration system with fleet configuration, such asconfiguring smart container operating units, selection of smartcontainer units that meet a freight storage and/or transportationservice order communication requirement, configuration and designationof deployment of fleet communication resources (e.g., co-locating aninter-smart container operating unit repeater device with the team), andother fleet and smart container configuration considerations. In anexample of such fleet configuration assistance, a freight storage and/ortransportation service order may indicate a preference to use specificsmart container operating units. The fleet configuration system mayquery the communication control system regarding adaptation capabilities(e.g., of the communication management system and/or certain fleetcommunication resources) to support the preferred smart containeroperating units.

In an example of communication management adaptability capabilities forsupporting diverse smart container operating unit communicationconfigurations, the communication management system 13010 may support afirst team of smart container operating units performing operations inusing a different radio frequency for wireless communication than asecond team of smart container operating units that are performingoperations in the same radio signal range as the first team of smartcontainer operating units; thereby mitigating the likelihood ofcross-radio interference. Further, the communication management system13010 may provide for reliable communication through use of redundancy,such as through dual radio systems, automatic channel selection (e.g.,local networking, cellular networking, mesh networking, long rangesatellite networking, and the like). Fleet communication resources mayinclude smart container operating units acting as network elements, suchas when smart container operating units are configured into one or moremesh networks and the like. Smart container operating units mayfacilitate communication in other ways, including visually, such asthrough use of light sources (e.g., Morse code or binary transmissions),physical gestures, infrared signals, ant-based trails, and the like.Auditory communications among smart containers (e.g., non-human languageencoded audio signaling), ultrasound and other auditory-based techniquesmay be rendered as a form of communication among smart containers. Muchlike how co-located smart containers on different teams may usedifferent radio frequency signals, co-located smart containers may usedifferent auditory signaling to assist in communication clarity amongteam members.

In embodiments, the communication management system 13010 may beconstructed as a plurality of independent communication systems that areconfigured to meet at least a corresponding portion of fleetcommunication needs. In an example, the communication management system13010 may be constructed with a first communication system forcommunicating among elements within the fleet management system 13002(or any other fleet system, system, module, team, fleet segment and thelike), and with a second communication system for communication amongintelligence service 13004 elements (or any other portion of the fleetsystem that can be separated from the first communication system), sothat disruption of any individual communication system may be isolatedfrom other system communication systems, thereby reducing impact ofcommunication problems throughout the system. Further in this example,the fleet management system 13002 and its constituent elements (e.g.,job configuration system 13018, and the like) may continue tocommunicate through the first communication system and indeed performall pertinent fleet operation functions (including communication withremotely deployed fleet smart container operating units and the like)even though access to intelligence service 13004 elements, such as amachine learning system, may be compromised due to problems with thesecond communication system serving the intelligence service 13004.Further, the communication management system 13010 may include securityfeatures that effect isolation and shunning of systems, sub-systems,system elements, communication systems, and other system resources thatappear to be compromised due to malware or the like. Other independentcommunication systems include smart container-to-smart containercommunication systems, robot-to-smart container communication systems,human-to-smart container communication systems, emergency responsecommunication systems, and the like. Yet further independentcommunication systems may be based on aspects, such as confidentialityof information (e.g., negotiations between a fleet management providerand a shipper), fleet operations oversight and the like. In embodiments,the communication management system 13010 may be constructed to providerole-based (or the like) access to different communication systems. Forexample, a fleet operations executive may be granted concurrent accessto smart container operating units allocated to different jobs forperforming fleet supervisory functions.

In addition to and/or instead of separated communication systems, thecommunication management system 13010 may provide for redundancy(multi-frequency radios, and the like) to address exception conditionsthat may cause network compromise, may require overriding operationalcommunication channels for emergency use, and the like.

In embodiments, the communication management system 13010 may providefleet resource-specific (e.g., individual smart container operatingunit) secure communication so that two fleet resources (e.g., two smartcontainer operating units, a smart container operating unit and a fleetmonitoring system, and the like) may communicate securely. Thecommunication management system 13010 may further provide broadcastcapabilities to support notification, update, alert, and other services.Broadcast capabilities may be fleet-wide (e.g., a notice to all fleetresources to observe daylight savings time), team-specific (e.g., anupdate to all team members regarding role changes of team members),job-specific (e.g., an alert to fleet resources assigned to a job, whichmay include a plurality of smart container teams, that the job is put onhold), fleet resource type-specific to address issues that concerncertain types of fleet resources (e.g., such as smart containeroperating units), fleet support units, location-specific units (e.g.,all units within a foreign jurisdiction), and the like.

In embodiments, the communication management system 13010 may use ormanage job-specific communications elements together with other fleetmanagement system features or services including, without limitation,the security system 13046, the network system 13202, and variousresources including Artificial Intelligence (AI) chipsets, dataencoders, communication spectrum frequencies, and the like. Thecommunication management system 13010 may work together with thesecurity system 13046, such as by providing secure high-up-time accessto fleet and associated communication resources. As an example, asecurity system 13046 may utilize a portion of configured communicationchannels (e.g., wired inter-computer links, wireless networks, and thelike) that may be reserved by the communication management system forsecurity use. The portion may include physically dedicated elements(e.g., wired connections, wireless access points that operate over adedicated set of frequencies, and the like). In embodiments, providingdedicated wireless access may include prioritization of security systemaccess to existing wireless networks, such as by routing security systemdata packets, streams, and the like ahead of non-security systempackets. As another example, a communication management system mayallocate communication devices with greater battery energy (highercharge) and/or fixed power supply for security system use whileallocating lower power, lower energy, and/or rechargeable devices fornon-security system use. Security system communication resourcemanagement and control may be fleet-wide, job-specific, team-specific,deployment locale-based, geolocation-based, and the like.

A further cooperative operation of security system 13046 with thecommunication management system 13010 may include managing access byfleet resources to external resources (e.g., websites, and the like) aswell as access by external resources to fleet resources. The securitysystem 13046 may deploy security agents and the like to fleet resourcesbased on allocation and/or configuration of those resources. As anexample, a firewall-type security function of the security system 13046may be deployed at, among other things, access points managed by thecommunication management system to connect distinct job-specificcommunication systems.

In embodiments, the communication management system 13010 may takeadvantage of intelligence capabilities of fleet resources, such asresources with artificial intelligence capabilities (optionally providedby AI-specific chips and chip sets and the like), to establish dynamiccommunication management functions that enrich and work with fleetsecurity capabilities to further reduce the likelihood of a successfulintrusion into a fleet communication system. As an example, AI-basedfunctionality deployed throughout at least portions of fleet resources(e.g., individual smart container operating units and the like) may berelied upon to detect local environments with increased risk ofintrusion or other threat (e.g., based on contextual and historicalinformation representative of such environments and the like) so thatthe communication management system, optionally in cooperation with thesecurity system 13046, may adapt fleet communication resources forreducing such risk.

The communication management system 13010 may make use of and/orfacilitate control of use by others of the network system 13202. As anexample of management of the network system 13202, the communicationmanagement system 13010 may treat the network system 13202 as a resourceto be managed for use by fleet resources for communicating, such as bydetermining and/or controlling which resources utilize the network, howresources using the network at the same time may be coordinated, networkloading limits for such resources, and the like.

In embodiments, the smart container system includes a remote-controlsystem 13012 that is configured to provide a framework for remotelycontrolling smart container operating units and other external resourcesto complete freight storage and/or transportation service orders. Inembodiments, the remote-control system 13012 may manage definition anduse of control signals for remote operation of smart container operatingunits, fleet support units, external resources and the like. Smartcontainer remote-control as enabled by the remote-control system 13012may include definition and management of local smart container operatingunit to smart container operating unit control signaling, such as when ateam supervisor smart container is directing one or more smart containerteam members to self-load onto a container ship. Other examples ofremote-control signal management may include smart container-to-smartcontainer fleet support signaling, intra-team smart container operatingunit signaling, and the like.

In embodiments, the remote-control system 13012 is constructed to assistthe order execution system 13022 and provide a framework for remotelycontrolling smart container operating units and other external resourcesto complete tasks and/or jobs. The remote-control system may managedefinition and use of control signals for remote operation of smartcontainer operating units, fleet support units, external resources andthe like. Smart container remote-control as enabled by theremote-control system 13012 may include definition and management oflocal smart container operating unit to smart container operating unitcontrol signaling, such as when a team supervisor smart container isdirecting one or more smart container team members to load and/or unloadcargo. Other examples of remote-control signal management may includesmart container-to-smart container fleet support signaling, intra-teamsmart container operating unit signaling, and the like. In embodiments,the remote-control system uses resources of the smart container system13000, including, for example, the communication management system13010, the security system 13046, and/or network system 13202 to accessinformation, in some cases make decisions, and execute commands. Theframework for remotely controlling smart container operating units maycomprise a series of actions-based standard rules, adapted rulesmodified by situational awareness, emergency rules, exceptions, humandecisions, ethical rules, the fleet intelligence system, etc. However,specialized, fall-over, or other communications necessary to handle arange of remote-control requirements may be part of the communicationmanagement system 13010 that may facilitate delivery of remote-controlcommunication and/or signaling what the communications should be versusmay be determined from use of the remote-control system 13012.

The remote-control system 13012 may recognize a plurality of initiatorsof remote-control signals, including local supervisor remote-controlinitiators, human (local or remote) remote-control initiators, automatedfleet-based remote-control initiators (e.g., fleet artificialintelligence system and the like), and third-party remote-controlinitiators (e.g., for law enforcement and the like). Remote controlsignaling may include managing remote control signals to fleet-externalresources, such as fire and emergency response resources, infrastructureresources, third-party smart container service providers, and the like.

The fleet resources that may participate in remote-control operationsmay be diverse in both implementation and protocols, such as oldergeneration smart container operating units, human fleet resources,quantum computing elements and the like. Therefore, the remote-controlsystem 13012 (in cooperation with the communication management system13010) may be constructed with knowledge of multiple remote operationalprotocol (multi-protocol) capabilities to ensure any two devicesexchanging control signals can do so reliably. In embodiments,multi-protocol capabilities may include handling and/or providing as aservice protocol-to-protocol translation, remote-control signalconsolidation and interpretation, protocol normalization, and the like.In embodiments, the communication management system 13010 may utilizethese protocol handling capabilities directly as noted above and by APIand the like, or by being configured with such protocol handlingcapabilities (e.g., deployed with protocol handling capabilities of theremote-control system 13012. In embodiments, the remote-control system13012 (or equivalent functions thereof integrated with the communicationmanagement system 13010) may rely on portions of the intelligenceservice 13004, such as digital twin and/or artificial intelligenceservice, to facilitate, for example, protocol translation and/oradaptation. Therefore, the remote-control system 13012 may providereal-time, on demand protocol translation, optionally assisted by thefleet intelligence system. The remote-control system 13012 may supportfleet-external remote-control via a port that is configured forintegration with external and/or third-party remote-controlarchitectures. Remote-control may be communicated via dedicatedinfrastructure and/or communication features (e.g., short-distancebroadcast capabilities).

Remote-control, such as control of smart container operating units, maybe initiated, at least in part, by a human operator. In embodiments, asmart container 13026 may encounter unexpected and/or unknown conditionsduring order execution (e.g., as may exemplarily be reported by theorder execution system 13022) and defer to a human operator to remotelycontrol smart container operating unit(s). Optionally, one or moreintelligence service 13004 components, such as an artificialintelligence system, may be referenced for at least candidateremote-control signals. In embodiments, an order execution plan mayindicate, at a predetermined operational task, that smart containeroperation should be guided by a human operator. When such a task isanticipated to occur in a job workflow (e.g., by a shipping executionmonitoring instance, such as a supervisor smart container and the like),the remote-control system 13012 may be called upon to oversee aremote-control connection between a suitable human operator and thesmart container, smart container operating units, team, team supervisor,and the like, executing the workflow that calls for human operatorcontrol.

In embodiments, the remote-control system 13012 may have access to a setof remote-control signal sequences for performing certain tasksremotely. The remote-control system 13012 may, based on context of aworkflow being performed, suggest to a human operator and/or anautomated control system one or more remote-control signal sequences. Inembodiments, the remote-control system may process input from a humanoperator (e.g., commands such as “stop”, “unload” and the like),optionally with help of other fleet resources (e.g., an artificialintelligence system and the like) and generate a set of remote-controlsignals for remotely controlling fleet resource, such as a smartcontainer operating unit and the like. Remote control signal sequencesmay be preconfigured for handling a range of real-time situations, suchas security breaches, equipment failure, and the like. In addition tofacilitating and/or managing remote-control of a smart containeroperating unit, remote-control signal sequences may be used forreconfiguration of fleet resources deployed and/or allocated for a job,task, workflow, and the like. A human operator (or an automated systemmonitor-type application) may provide remote control signals that arecommunicated to the viable members of the team to adjust task roles andactions accordingly, such as by communicating a remote-control signal toone or more of the viable members to communicate with a smart containeroperating unit configuration server to receive reconfigurationinstructions and reconfiguration data.

Although generally described herein as remote-control signals, theremote-control system 13012 may facilitate remote-control by arrangingremote control signals into remote control instructions (e.g.,combinations of remote-control signals, abstractions thereof and thelike) at the fleet level, team level, smart container level and thelike. As an example of remote-control instruction functionality, theremote-control system 13012 may receive input, such as from a humanoperator desiring to instruct a smart container to drive up a ramp ontoa container ship. In this example, the remote-control system may receivethe human operator remote-control instruction, adapt that instructioninto one or more different remote-control signals for the smartcontainer 13026, generate corresponding remote-control signals, andensure communication of those signals (e.g., via the communicationmanagement system 13010 resources) to the smart container 13026 to beremotely controlled by the human operator.

Smart container operating unit responsiveness to aggregatedremote-control signals (e.g., instructions or set of instructions) maybe based on a wide range of fleet intelligence capabilities, knowledge,priorities, goals, and the like. In general, use of system-based and/orsmart container operating unit-based artificial intelligencecapabilities supports wider independent decision-making capabilities forindividual smart container operating units with greater contextualgravity.

In embodiments, the remote-control system 13012 may integrate securityfeatures to thwart takeover, compromise, misuse, or interference withcontrol of remotely controlled smart container operating units.Resources used by the remote-control system 13012 (e.g., data storageresources, computing resources, remote-control system state data, andthe like) may be configured with security features, such as encoding,decoding, packetizing, and the like. Further, the remote-control system13012 may include and/or support control override capabilities thatenable a human operator (for example) to securely gain remote-control ofa smart container that is otherwise not directly engaged withremote-control signaling or operating independently of remote-controlsignals, such as, autonomously, collaboratively with other smartcontainer operating units and the like.

In embodiments, the smart container system 13000 may include a humaninterface system 13038 that provides a human interface that allows usersto access the smart container system 13000 and/or individual smartcontainer operating units (e.g., for remote control) from a remotedevice (e.g., a user device, a VR device, an AR device, and/or thelike). In embodiments, the human interface system 13038 facilitatesfreight storage and/or transportation service order entry (including anyjob-related parameters), fleet operations management, fleet resourcemanagement, fleet computing system, software and data structuremanagement (e.g., system upgrades and the like), human access to smartcontainer operating units (e.g., for remote control of a smart containeroperating unit), augmented and/or virtual reality visualizations offleet operation, and data extraction (e.g., for generation of and/orvalidation of smart contracts associated one or more freight storageand/or transportation service orders and the like). As an example of useof a human interface system 13038, a user may access status updates of arequested job via the human interface system 13038. The user may use aremote device to observe smart container operating units performingtasks for the requested job. In this example, the human interface system13038 may interact with other fleet components, such as the orderexecution system 13022, to direct image capture resources (e.g.,camera-based overhead drones) to provide images of smart containeroperating units assigned to and currently performing job tasks.

In embodiments, the fleet management system 13002 may include a jobconfiguration system 13018, a fleet configuration system 13020, aresource provisioning system 13014, a logistics system 13016, and anorder execution, monitoring, and reporting system 13022 (also referredto as a “order execution system” 13022).

In embodiments, the fleet management system 13002 includes a resourceprovisioning system 13014 that manages provisioning resources for smartcontainer operating units in a fleet, such as provisioning resources forsmart container teams, smart container fleets, smart containers, and/orsupporting resources (e.g., edge devices, communication devices,container ships, cranes, additive manufacturing systems (e.g., 3Dprinters), and the like). In embodiments, resources may include physicalresources, digital resources, and/or consumable resources. Examples ofphysical resources may include, but are not limited to, endeffectors/manipulators, environmental shielding components, sensorsand/or sensor systems, companion resources (e.g., drones, robots,container ships, trucks, railway systems, cranes, lifts, and the like),hardware resources (e.g., specialized processing modules, data storage,networking modules, tethering modules, and the like), spare parts, humanresources (e.g., technicians, operators, and the like), power sources(e.g., generators, portable batteries), and the like. Non-limitingexamples of digital resources may include software, operatingparameters, job-specific data sets, and the like. Non-limiting examplesof consumable resources may include fuel, packaging supplies, weldingsupplies, washdown/cleanup supplies, and many others.

In embodiments, the resource provisioning system 13014 may provisionphysical resources from an inventory of physical resources, such asfleet-specific inventories, regional public-use inventories,rental/per-use fee-based resource inventories, on-demand resourceproduction systems (e.g., 3D printing of end effectors and the like),third party inventories, and the like.

In embodiments, the resource provisioning system 13014 may workcooperatively with other systems of the fleet operations system, such asfleet configuration systems, fleet resource scheduling and utilizationsystems, and the like to ensure fleet resource provisioning rules arefollowed. Physical resources to be provisioned may also includecomputing resources, such as on-smart container computing resources,smart container operating unit-local fleet-controlled computingresources, cloud/third-party based computing resources, computing andother modules and chips (e.g., for deployment with/within a smartcontainer operating unit), and the like. In some embodiments, the fleetresource provisioning rules may be defined in governance standardslibraries, such that the resource provisioning system 13014 interfaceswith the intelligence service to ensure that provisioned resourcescomply with the provisioning rules.

In embodiments, digital resources to be provisioned by the resourceprovisioning system 13014 may be provisioned through fleet configurationcapabilities, such as software/firmware update pushing (e.g., to updatea smart container's on-board software), resource access credentialing(e.g., to access network resources, such as job-specific smart containerconfiguration data and the like), on-smart container data storageconfiguration/allocation/utilization data, and the like. Use of aprovisioning system 13014 may include provisioning equipment, material,software, data structures, and the like that are made and/or sourcedspecifically for a given freight storage and/or transportation serviceorder.

In embodiments, the provisioning system 13014 may further operatecooperatively with contract systems, such as third-party smart contractsystems, and the like. In some embodiments, a freight storage and/ortransportation service order may reference or comprise a smart contractthat may include and/or result in configuration of an instance of theprovisioning system 13014 that is compliant with the request. As anexample, a provisioning system 13014 may receive, such as from a jobconfiguration system 13018, smart contract terms that call outprovisioning constraints and/or guidance. The provisioning system 13014may interpret these contract terms, thereby producing a set of fleet andconsumable resource provisioning constraints.

While the examples described above for a resource provisioning system13014 generally focus on order execution-related provisioning, theresource provisioning system 13014 may further handle provisioning offleet resources, such as computing resources, access to and/or executionof fleet elements, such as a fleet configuration system, intelligenceservice, and the like. In embodiments, provisioning of certain resourcesmay be enacted as part of a negotiation workflow for acceptance of afreight storage and/or transportation service order. As an example,provisioning certain intelligence services (e.g., a fleet levelintelligence service) may result in a higher charge to a shipper thanother intelligence services (e.g., only a smart container-levelintelligence service being deployed smart container operating units). Asnoted above and elsewhere herein, intelligence services can bring valueto the fleet and job configuration functions of the system; therefore,provisioning such systems as part of a freight storage and/ortransportation service order negotiation may justify the additional costto the shipper. In some scenarios, prioritization of the systemresources, such as a fleet configuration system, may impact provisioningsystem 13014 functions.

In embodiments, the fleet management system 13002 includes a logisticssystem 13016 that handles, among other things, logistics planning andexecution for meeting shipment requirements, maintaining smartcontainers, maintaining availability of fleet resources (smart containeroperating units, physical resources, and the like), and pickup anddelivery of parts (e.g., replacement parts, end effectors, supplies, andthe like). In embodiments, the logistics system 13016 can leverageintelligence services, such as machine learning systems and/orartificial intelligence systems to recommend logistics plans.

Logistics plans may refer to a workflow that is generated to result inthe delivery of a set of items to a particular location. In embodiments,the logistics system 13016 may generate logistics plans that utilizefleet resources (e.g., smart containers, container ships, robots,trucks, cranes, railways, or the like) for execution of a logisticsplan. In embodiments, the fleet operation system 13002 may leverage the(system-level) intelligence service 13004 to assist in logisticsplanning and decision-making.

In embodiments, the fleet management system 13002 includes a maintenancemanagement system 13028 that may be configured to schedule andeffectuate maintenance for fleet resources, such as smart containeroperating units. A maintenance management system 13028 may handle fieldmaintenance needs and requests, including scheduled maintenance of fleetrecourses in the field to mitigate impact on smart container operatingunit utilization due to travel from a deployed job site to a repairdepot. The maintenance management system 13028 may also coordinatemaintenance and repair operations at repair depots, and the like. Inembodiments, a maintenance management system 13028 may include, provideaccess to, and/or be integrated with mobile maintenance vehicles, spareparts depots, third-party maintenance service providers and the like. Inembodiments, maintenance needs for fleet resources housed in storageareas, such as warehouses, remote inventory depots and the like may beevaluated by the maintenance management system 13028 for pre-scheduledmaintenance, such as when a preventive maintenance activity for a smartcontainer is upcoming so that the smart container is less likely torequire maintenance during a deployment.

In embodiments, the maintenance management system 13028 may monitor thestate of the fleet resources, such as smart container operating units,via resource state reports that may be provided on a scheduled basis orin response to an inquiry for smart container operating unit state bythe maintenance management system 13028 and the like. In embodiments,the maintenance management system 13028 may monitor smart containeroperating unit communication for an indication of a potential servicecondition, such as a smart container operating unit signaling that it isexperiencing reduced power output, a smart container operating unitreporting exposure to certain ambient conditions (e.g., excessive heat),a smart container operating unit reporting a leak involving liquid cargothat requires cleaning, a lack of heartbeat signal from a smartcontainer operating unit to a smart container health monitor resource,and the like. Further, a maintenance management system 13028 may deployprobes within smart container operating and/or supervisory software thatmay perform maintenance management functions on a smart containeroperating unit, such as monitoring information in a smart container datastore that stores smart container operating unit state information,activating self-test operating modes, collection of data that providesindications of smart container maintenance needs, and the like. Yetfurther, a maintenance management system 13028 may include maintenancerobots that may be deployed with smart containers in a team of smartcontainer operating units for performing a requested job.

A maintenance management system 13028 may be constructed to takeadvantage of a range of system services and capabilities to schedule andeffectuate maintenance, including leveraging human/operator input (e.g.,a human observer may indicate that a smart container operating unitappears to be operating erratically), smart container process automationof maintenance activities, artificial intelligence for predictingmaintenance instances for scheduling, machine learning to help identifynew opportunities for scheduling and performing maintenance (e.g.,analyze performance of smart container operating units that have beenmaintained for certain conditions before operating under thoseconditions, such as upgrading the cooling system of a smart containerbefore operating in a high temperature environment), and the like. Inembodiments, a maintenance management system 13028 may receivemaintenance-related input. In embodiments, candidate sources ofmaintenance related input may include human operators/observers,maintenance scheduling services, third-party service providers, smartcontainer production vendors, and parts providers to schedulemaintenance. The maintenance management system 13028 may also leveragebusiness rules (e.g., rules established for a team, fleet, by a shipper,determined by a regulatory agency, and the like), association tables,data sets, databases, and/or maintenance management libraries todetermine appropriate maintenance workflows, service actions, neededparts, and the like. In embodiments, a maintenance activity may beassigned by the maintenance management system to a fleet resource, suchas a maintenance smart container, a human technician, a third-partyservice provider, and the like.

In embodiments, smart container operating units that are deployed may beconfigured with one or more maintenance protocols to perform, amongother things, self-maintenance, such as self-cleaning, calibrating endeffector operations, and the like. Self-maintenance may include, withoutlimitation, reduction in capabilities responsive to detection of acompromised smart container operating unit feature, such as a faulty 3Dprinting system or faulty systems for securing cargo (e.g., steelstrapping, polyester strapping, dunnage bags, or the like). A deployedsmart container operating unit may determine that a capability iscompromised and, optionally with support of the maintenance managementsystem 13028, may switch assignments with another smart container sothat the compromised capability can be resolved when time permits ratherthan causing a delay in completion of a shipment. Also, smart containeroperating unit intelligence (e.g., on-smart container AI and the like)may predict a compromise in smart container capabilities based on, forexample, time-to-failure data for the smart container capability.

In embodiments, the maintenance management system 13028 may leverage theintelligence service 13004 (e.g., the system level intelligence service13004) to predict when maintenance may be performed for smart containeroperating units and/or components thereof. In some of these embodiments,the maintenance management system 13028 may request a digital twin of asmart container operating unit from the intelligence service 13004. Inthese embodiments, the digital twin may reflect a current condition ofthe smart container operating unit, such that the smart containeroperating unit digital twin may be analyzed to determine whethermaintenance is required for the smart container operating unit.Additionally, or alternatively, the digital twin service of theintelligence service 13004 may run one or more simulations involving thesmart container operating unit to predict when maintenance may berequired. In some of these embodiments, outputs of the digital twin ofthe smart container operating unit may be analyzed (e.g., using amachine-learned prediction model or a neural network) to predict if/whenmaintenance may be required.

In embodiments, the fleet management system 13002 includes a jobconfiguration system 13018. In embodiments, a shipment configurationsystem receives freight storage and/or transportation service orders,such as from customers that book a smart container shipping service. Inembodiments, a freight storage and/or transportation service order mayindicate a set of freight storage and/or transportation service orderparameters. Non-limiting examples of freight storage and/ortransportation service order parameters may include: timingrequirements, origin and destination of shipment (e.g., region, address,coordinates, or the like), number of smart containers required, type ofcontainers required (e.g., tank containers, bulk containers, 20-ftstandard containers, 40-ft high-cube containers, or the like), containerutilization requirements (e.g., full container (FCL) vs. sharedcontainer (LCL)), cargo descriptions (e.g., number of packages, totalvolume, or total weight), whether the cargo includes personal effects,other required tasks (e.g., inspection tasks, packaging tasks,unpackaging tasks, unloading tasks, loading tasks, 3D printing tasks,growing tasks, assembling tasks, monitoring tasks, or the like), pricinginformation, and any other suitable parameters. In embodiments, thefreight storage and/or transportation service order parameters which maybe indicative of what types of smart container operating units areneeded and/or functionalities thereof. These and other freight storageand/or transportation service order details are described elsewhereherein.

In embodiments, quantum optimization may be enabled by a quantumcomputing service 13008 that may optimize assignments across fleetresources, such as smart container operating units and the like. Aquantum computing service 13008 may further optimize routing (logical,physical, and electronic) associated with smart container fleets,shipments, team, communications, logistics and the like. Additionally,or alternatively, in some embodiments a quantum computing service 13008may be employed to optimize combinations of smart container resourceswith other resources across a variety of fleet functions, includingenergy consumption, computational capacity and utilization,infrastructure resource planning, engagement and utilization, riskmanagement, computing storage capacity, and the like. The quantumcomputing service 13008 may also be used for optimizing smart containerdesign, optimizing smart container services pricing, optimizing smartcontainer charging (e.g., optimizing the route of a smart containerhaving solar panels such that it receives sufficient levels ofsunlight), or the like.

In embodiments, a job configuration system 13018 and other fleetresources (e.g., fleet configuration, system intelligence, smartcontainer operation and the like) may benefit from use of deep learningtechniques for task, workflow, and order execution plan optimization aswell as for learning, among other things, from failures. In theseembodiments, the job configuration system 13018 may request deeplearning services from the system-level intelligence service 13004,which leverages neural networks and/or other machine-learned models todetermine job configurations based on a set of features, includingfeatures extracted from a freight storage and/or transportation serviceorder. In these embodiments, the artificial intelligence services may beconfigured to learn task workflows, job configurations, and the like.

In embodiments, job configuration, fleet configuration (which mayinclude smart container configuration), and/or as order execution mayfurther enhance fleet functions, performance, and outcomes through useof local context-adaptive task assignment, execution, resource routingand the like. This adaptive capability may be further enabled throughpeer-to-peer based communication (e.g., smart container operating unitswithin a team) that reveals context of job activities rapidly andefficiently.

In embodiments, artificial intelligence for automation of smartcontainer assignment and execution (e.g., smart container processautomation through learning) may function cooperatively with elements ofthe smart container system 13000, such as a fleet management system13002 and intelligence service 13004, to learn smart containerassignment from, for example, human operator assignment activity. Otherlearning that an artificial intelligence system may yield in context ofsmart container fleet configuration and operation may be based onoutcome measures of success, including task completion, time tocompletion, percentage of damage-free shipments delivered, cost ofcompletion, quality of completion, ROI for resources, resourceutilization, and others.

These and other job configuration details, including operational flowsof the job configuration system 13018 are depicted and described inrelated figures herein.

In embodiments, the fleet management system 13002 includes a fleet andsmart container configuration system 13020 (also referred to as fleetconfiguration system 13020) that may work cooperatively with a jobconfiguration system 13018 to determine configurations of fleetresources (e.g., smart container operating units, teams, and the like)to satisfy freight storage and/or transportation service orders from aplurality of concurrent and/or overlapping freight storage and/ortransportation service orders. The fleet configuration system 13020 maydetermine fleet and smart container configurations based on freightstorage and/or transportation service orders, required tasks, budget, atimeline, availability of smart containers or smart container types,availability of container ships or other modes of transport, traffic atcontainer terminals or ports, and/or other suitable considerations. Insome embodiments, the fleet configuration system 13020 may leverage thesystem-level intelligence service 13004 to determine fleet and/or smartcontainer configurations. In some of these embodiments, the intelligencerequest may include a proposed fleet configuration and other relevantdata (e.g., cost constraints, cargo type, origin location, destinationlocation, route environments, etc.). In response, the intelligenceservice 13004 may output a proposed fleet configuration. Further detailsof a fleet configuration system 13020 are described and depicted infigures elsewhere herein.

In embodiments, a fleet management system 13002 may include an orderexecution, monitoring, and reporting system 13022 (also referred to asan order execution system 13022). An order execution system 13022 mayreceive an order execution plan from the job configuration system 13018that it processes by coordinating activities of system functions, suchas logistics for smart container and fleet resource delivery, dataprocessing system 13024 allocation for facilitating data collection,cataloging, library management and data processing activities for orderexecution. In general, the order execution system 13022 may start a jobwith committing and managing resources, including resources beyond thoseconfigured by the job configuration system 13018, such as computing,storage, bandwidth, and the like as may be defined by and/or determinedto be useful for executing the order execution plan.

In embodiments, the order execution system 13022 may further facilitateadherence to reporting requirements (e.g., shipment-specific,fleet-specific, compliance-related reporting, and the like) associatedwith order execution. In embodiments, reporting may include datacollection (e.g., from smart container operating units, sensor systems,user devices, databases, and/or the like), data processing, and feedbackpreparation for use of order execution data by job and fleetconfiguration systems and the like. In embodiments, the order executionsystem 13022 may be assisted by other system capabilities that transmit,process, store, and manage data that impacts order execution, such asthe maintenance management system 13028, the resource provisioningsystem 13014, and the communication management system 13010 thatfacilitates communications among smart container operating units, teams,and fleets, and others. These and other fleet and external resources mayprovide information to the order execution system 13022 for facilitatingoperational aspects of a requested job, such as which communicationresources has the communication management system 13010 reserved and/orallocated for the requested job, service and/or maintenance requirementsfor smart container operating unit and other resources being used toexecute a job, changes to resource provisioning that occur afteroperation of a job has commenced, and the like.

In embodiments, the order execution system 13022 may further facilitateevaluation and modification of an order execution plan while executingthe job by, for example, identifying bottlenecks that are developing dueto on-the-job conditions (e.g., heavy container port traffic, groundconditions not as expected due to excessive rain, and the like).

In embodiments, the order execution system 13022 may perform a varietyof data pipeline functions during execution of a job. In embodiments,data pipeline functions may include, among other things, optimizing useof preconfigured sensor and detection packages that combine sensorselection, sensing, information collection, preprocessing, routing,consolidation, processing, and the like. In embodiments, sensor anddetection packages may be activated by the order execution system 13022when use thereof is indicated as serving a range of monitoring/reportingactivities. Other data pipeline function examples include optimizingon-smart container storage, selective sensor data filtering for reducedimpact on communication bandwidth (e.g., reducing the demand forwireless network utilization), exception condition detection andpipeline adaptation/data filtering, and others.

In embodiments, the order execution system 13022 may monitor, and ifnecessary, address smart container power demand during order execution.In these embodiments, the order execution system 13022 may ensure, forexample, battery charge capacity (or other energy source levels, such asfuel levels) across multiple smart container operating units to meet jobtask and workflow requirements, such as a queue of tasks that should notbe interrupted. In embodiments, smart container power demand managementmay include fleet, team, and individual smart container operating unitrouting to complete tasks with reduced delays in overall productivitywith integrated smart container charging activities. Further details ofthe functions and operation of the order execution system 13022 aredescribed throughout the disclosure.

In embodiments, smart container functionality, including during orderexecution, may be combined with 3D printing services and systems toenable, for example, agile, remote, flexible manufacturing on anas-demanded basis through, for example, deployment and use of optionallyautomated smart container 3D printing and production capabilities forlast-mile customization of products.

In embodiments, the order execution system 13022 may execute, deploy,and/or interface with a set of smart contracts that monitor and reporton smart container operating units 13040. In embodiments, robustdistributed data systems, such as distributed ledgers (e.g., public orprivate blockchains) may be utilized for tracking and enhancing smartcontainer fleets and/or smart container activities, as well asallocation of smart container resource utilization cost to relevantparties. In some of these embodiments, the distributed ledger nodesstore and execute smart contracts. In embodiments, the smart contractsmay be configured to monitor freight storage and/or transportationservice orders, order execution, resource use, and/or the like. Forexample, in some embodiments, smart container operating units may beconfigured to provide evidence of completion of a task (e.g., a deliveryof cargo) to a smart contract, such that the smart contract may triggeractions (e.g., payments, recordation, or the like) in response tocompleted tasks. In another example, smart container operating units maybe configured to report location data, sensor data, status data (e.g.,charge levels, cargo condition and/or status, or the like), and/or othersuitable data, whereby the smart contract may be configured to triggercertain actions based on the received data.

In embodiments, a smart container system 13000 may include a dataprocessing system 13024 that may provide, among other things, access toscalable computation capabilities for any smart container freightservice operations and/or intelligence resources, data managementcapabilities (e.g., data caching, storage allocation and management andthe like), access to and control of fleet and/or job-related datastores, such as libraries, fleet resource inventory control andmanagement data structures and the like.

In embodiments, the fleet management system 13002 may provide supportfor satisfying freight storage and/or transportation service orders. Forexample, the components of the fleet management system 13002 mayfacilitate resource provisioning and logistics to ensure that fleetresources (e.g., smart container operating units, physical modules,and/or support devices) are provided in an efficient manner to satisfythe freight storage and/or transportation service order, such as timingof order execution and the like. For example, in some embodiments, thefleet management system 13002 may employ “just-in-time” strategies tofacilitate delivery of fleet resources and/or maintenance tasks toensure fleet resources are allocated in an efficient manner withoutsignificantly impacting job completion times. In some of theseembodiments, the fleet management system 13002 may leverage theintelligence services to anticipate the fleet resource needscorresponding to various freight storage and/or transportation serviceorders and/or order execution plans to anticipate the fleet resourceneeds and to arrange for delivery and/or maintenance of such fleetresources.

In embodiments, the order execution system 13022 may anticipatejob-related resource needs in a job-specific manner to predict whenspecific resources will be required for a specific job. For example, theorder execution system 13022 (working in combination with theintelligence service) may generate a schedule of in-progress and/orupcoming tasks for a specific freight storage and/or transportationservice order, and in response, may determine when certain fleetresources are likely to be needed and/or to come available.Additionally, or alternatively, the order execution system 13022 maypredict the job-related resources for a specific job in other suitablemanners. For example, prediction of resource needs may be determinedbased on a pattern of fleet resource needs as derived from a freightstorage and/or transportation service order history of the shipper; aresource usage history of the shipper from the previous N jobs performedfor the shipper; timing of freight storage and/or transportation serviceorders (e.g., orders are typically received on a Thursday for jobs tostart on Monday the following week), and/or the like. Businessrelationships among entities can form a basis for predicting fleetresource needs and timing of the shipper/buyer based on actions,including freight storage and/or transportation service orders, of thesupplier/seller/consumer.

In embodiments, many other factors may impact fleet resource needpredictions, such as weather forecasting and seasonal affects. Fleetresource need prediction may also be activated by events outside of thecore freight storage and/or transportation service order process, suchas natural disasters, accidents/emergencies, pandemics, and the like. Inanother example, other sources of information that may impactanticipation of fleet resource needs may include business goals andobjectives, such as reducing or increasing spending near the end of afinancial reporting period (e.g., a fiscal quarter, year, etc.). Anindication that a target shipper intends to cut back on expenses duringthe last few weeks or months of a fiscal reporting period may suggestthat fleet resources that are typically allocated to freight storageand/or transportation service orders by the target shipper will beavailable for other actions, such as maintenance, upgrading, allocationto other shippers and the like. In embodiments, fleet goals orobjectives may also impact fleet resource anticipation and thereforecorresponding preparation activities and the like. One such example is arequired upgrade of a class of smart container. In anticipation ofneeding to reserve the smart containers in this class, the fleetconfiguration functions may allocate alternate smart container typesthat can be reconfigured to satisfy the requirements of the reservedsmart container class for the duration of the upgrade activity.

In embodiments, anticipation of fleet resource needs may be determinedthrough use of smart container system 13000, such as the intelligenceservice 13004 and the fleet management system 13002. For example, insome embodiments the intelligence service 13004 may analyze sources ofdata that may impact fleet resource demands, such as weather forecasts,public activity calendars, freight storage and/or transportation serviceorder data (e.g., timing, job parameters, relations to other freightstorage and/or transportation service orders, and the like), socialmedia activity, government activity and/or legislation, and the like. Inthis example, the intelligence service 13004, acting in cooperation withthe fleet management system 13002, may predict fleet resource demandbased on an analysis of the disparate data sources (e.g., using a neuralnetwork or the like). In these embodiments, the intelligence service13004 may process the data from the disparate data sources and determinea likelihood of fleet resource needs across a range of factors.

In embodiments, a smart container system 13000 may interface withexternal data sources 13036 for performing various system functionsincluding job configuration, fleet configuration, job negotiation (e.g.,via a smart contract facility), order execution and the like. Examplesof external data sources for use by the system include value chainentities (e.g., third parties paying for shipping services and thelike), enterprise resource planning systems (ERPs) that may provide jobcontext for performing team configuration and/or execution of arequested job, smart contracts, and the like. Other external datasources may include third-party sensor systems (e.g., GPS data, valuechain logistics data, and the like) as well as third-party data streams(e.g., weather, traffic, electricity pricing, and the like).

In some embodiments, the smart container system 13000 may support theuse of smart contracts in relation to freight storage and/ortransportation service orders, job performance, resource allocation,and/or the like. In embodiments, freight storage and/or transportationservice orders may be routed through a smart contract handler thatcaptures job requirements, requestor goals and objectives, and fleetorder execution constraints into a dynamic smart contract. In someembodiments, smart contracts may be utilized throughout a smartcontainer fleet management system to address all manner of fleetoperations, such as administering negotiated routing of a smartcontainer from a first location (e.g., an origin location, containerport or terminal, a temporary storage/service location) to a secondlocation (e.g., a destination location, a container port or terminal, orthe like). As a further example, a smart contract may be put in place asa control for a bidding system for smart container time/taskutilization. As another example, a smart contract may monitor certainactivities (e.g., task-related activities and the like) relating to afreight storage and/or transportation service order. The smart contractmay rely on and/or benefit from access to fleet system data, (e.g.,route progress, sensor data, and the like) to trigger actions defined bythe smart contract, such as payments upon completion of a delivery ofsmart container cargo. The smart container system 13000 may provideaccess to fleet resources, including fleet data through ApplicationProgramming Interfaces (APIs), infrastructure elements such as sensornetworks, edge computing systems, and the like for updating statesrelevant to smart contract terms and conditions.

In embodiments, the job configuration system 13018 and the fleetconfiguration system 13020 collectively generate an order executionplan, according to some embodiments of the present disclosure. Inembodiments, an order execution plan may define a smart container routeand/or set of tasks that are to be performed in completion of arequested job and may further define a configuration of a fleet of smartcontainer operating units that are to complete the job. In embodiments,an order execution plan may include task definitions (which may includeroute definitions), workflow definitions, fleet configurations (whichmay include smart container configurations of individual smartcontainers), team assignments, and references to (or incorporation of)contextual information, such as container port site details and thelike. In embodiments, the job configuration system 13018 receives anorder that defines the job to be done and the job configuration system13018 may determine a set of task definitions that respectively define ashipping service and/or tasks that are performed by a smart container incompletion of a job. In embodiments, the job configuration system 13018further defines a set of workflow definitions. The workflow definitionsdefine at least one order in which tasks are performed in completion ofa project and/or job, including any loops, iterations, triggeringconditions, or the like. In embodiments, the job configuration system13018 may determine the workflows based on the task definitions thatcomprise a job. The job configuration system 13018 may leveragelibraries of preconfigured workflows to complete certain jobs.Additionally, or alternatively, the job configuration system 13018 mayleverage the intelligence service 13004 to obtain an initial workflowdefinition for a job and/or project that is part of a larger job. Insome embodiments, a human may configure the initial workflow definitionand/or may provide input that is used to determine the initial workflowdefinition. In embodiments, the job configuration system 13018 mayinterface with one or more components of the smart container system13000 to exchange information for developing a smart container fleetorder execution plan and/or to leverage one or more services thereof.For example, the job configuration system 13018 may interface with thedata processing system 13024, a smart container configuration library ofsmart container, fleet, project, and task related information, thefleet-level intelligence service 13004, the fleet configuration system13020, and the like.

In embodiments, the job configuration system 13018 may include aplurality of systems that perform order execution plan preparationfunctions by processing the information received in the freight storageand/or transportation service order. In embodiments, the systems of thejob configuration system 13018 may include a job parsing system, a taskdefinition system, a workflow definition system, and a workflowsimulation system. In the illustrated example, the job configurationsystem 13018 systems work in combination to generate an order executionplan that is used to define a set of smart container operating unitassignments. In embodiments, smart container operating unit assignmentsmay be supplemental to or integrated with an order execution plan andmay identify specific smart container teams and/or smart containersassigned to respective tasks. For example, smart container operatingunit assignment may define specific tasks and, for each task, mayidentify a specific smart container assigned to a task via a smartcontainer unique identifier and/or a specific smart container team witha team identifier assigned to the task. In embodiments, the smartcontainer operating unit assignments may be generated by the jobconfiguration system 13018 and/or the fleet configuration system 13020.

In embodiments, a job parsing system receives and parses a freightstorage and/or transportation service order to determine a set offreight storage and/or transportation service order parameters that areultimately used to determine a job definition, project definition(s),task definitions, workflow definitions, fleet configurations, and smartcontainer configurations. In embodiments, a job parsing system mayreceive a freight storage and/or transportation service order from auser via a user interface, such as human interface system 13038 thatreceives input by an operator to configure, adapt, or otherwisefacilitate parsing of the freight storage and/or transportation serviceorder. Additionally, or alternatively, the job parsing system mayreceive the freight storage and/or transportation service order from aclient device associated with a requesting organization.

In embodiments, the job parsing system may be configured with aningestion facility for receiving electronic versions of job descriptionsand related documents, such as GPS data, smart contract data and/orterms, links to the same, and the like. The ingestion facility may parsedocuments for keywords, references to activities and the like that canbe useful for determining job requirements. Further, keywords in theingested job content, such as weight terms, volume terms, routeenvironment terms, and the like may be usefully applied by the jobconfiguration system 13018 elements by providing insight as to thetype(s) of smart containers needed and the configurations thereof. As anexample, a keyword that suggests content to be moved weighs 25 tons,suggests a smart container transport device/team that has at least thatamount of moving capacity.

In embodiments, the job parsing system may incorporate and/or utilizemachine learning functionality (e.g., as may be provided by theintelligence service 13004) to improve techniques for parsing jobcontent which may include description data. In addition to machine-basedlearning from human-generated feedback on job content parsing results,learning may be based on experience with other job content parsingactions (e.g., prior freight storage and/or transportation serviceorders), common and special knowledge bases (such as technicaldictionaries), expert humans, and the like.

In embodiments, parsing of job content may include automated parsing ofstructured and unstructured text. In some embodiments, the job parsingsystem may be configured to identify (and optionally resolve)missing/unclear data and qualified job content data (collectivelyreferred to as “insufficient information”). In response to identifyinginsufficient information, the job parsing system may generate andprovide a request to a human operator via a user interface forclarification with respect to the insufficient information. Such arequest may identify specific inputs from the user to provide, such thatthe request identifies the clarifying content that was missing orunclear initially. Additionally, or alternatively, the parsing system13214 may determine the clarifying content from (e.g., through a queryof) a library 13044 that maintains data from prior freight storageand/or transportation service orders, such that the clarifying contentmay be obtained using the prior freight storage and/or transportationservice order information and context from the request. If the parsingjob is unable to determine the clarifying content, the parsing system13214 may generate a request for clarifying content, as discussed above.

In embodiments, a range of job description information may be providedto, determined, and/or extracted by the job configuration system 13018.Examples of freight storage and/or transportation service orderparameters may include, but are not limited to: origin location anddestination location information and/or other physical locations along aroute; weight requirements; volume requirements; cargo descriptions;smart container type(s); the number of smart containers required; modeof transportation (container ship, truck, rail, self-driving smartcontainer, hyperloop, and the like). 3D printing requirements (such asfor last-mile customization); digital data for environment layouts ofships, shipping container ports, shipping container storage facilities,and the like, such as 3D CAD models or scans may be available or mightbe completed as part of initial job scoping and may be used toautomatically provide task priority and workflow routing, smartcontainer selection, supervisory needs, etc.; operating environment(such as along a route) including temperature, hazard description(s),terrain, weather, etc.; deliverables, such as data, reports, analysis,and the like; customer interfaces for data exchange, such as networkinterfaces, APIs, security; communication network availability, such asland line, 4G, 5G, Wi-Fi, private networks, satellite, connectivityconstraints, and the like; budget constraints; timing requirements,including scheduling for port availability, scheduling for shipavailability, earliest start time, latest finish time, rate of activity,such as the number of smart containers active at any given time, and thelike. Examples of other job description information that may be handledby a job parsing system may include contract-related information, suchas smart contract terms, certification level of smart containeroperational software for deployed smart containers, insuranceprovisions, regulatory requirements (e.g., customs requirements), siteaccess requirements (e.g., a particular container port can be accessedonly when humans are present or only through coordination with humansthat are present on the site), conditions for assigning a proxy for atask, activity, workflow, or the entire job.

In embodiments, the job configuration system 13018 systems (e.g., jobparsing system, task definition system, and workflow definition system)may reference a library 13044 to identify content and structural filtersfor distinguishing smart container automation job content from other jobcontent (e.g., cost, payment, financing, etc.), preconfigured candidatetasks, workflows, and/or completed job configurations that substantiallymeet the requirements of the freight storage and/or transportationservice order. In embodiments, the library 13044 or another jobconfiguration library may facilitate mapping indicia of the job contentwith target terms that indicate smart container automation. As anexample of a use of an automated task from the library 13044, arequested freight transportation service may include a requirement formeasuring the temperature inside the smart container using a set oftemperature sensors. The job parsing system may identify the temperaturemeasurement requirement, and in response, the task definition system mayidentify an automated measurement task for measuring containertemperature in the library 13044 that meets the requirements of thatportion of the freight storage and/or transportation service order,which may be used in defining the order execution plan. If jobconfiguration system 13018 determines that a suitable job configurationis available (e.g., from the library 13044), such as if the freightstorage and/or transportation service ordered had previously beenrequested, the job configuration system 13018 may use a previous jobconfiguration corresponding to the previously requested job as aproposed job configuration for further validation with current fleetstandards and the like. For example, the intelligence service 13004 mayanalyze the proposed job configuration (e.g., with one or moreintelligence services, including without limitation a machine learningservice) with respect to a set of governance standards to ensure thatthe proposed job configuration comports with said standards. Theintelligence service 13004 may perform other intelligence-based taskswith respect to the proposed job configuration.

In some scenarios, the job configuration system 13018 may determine thatone or more tasks, workflows, routines, and the like do not have asuitable counterpart in the library 13044. In such a scenario, the jobparsing system may generate a data set that includes smartcontainer-fleet focused requirements (e.g., task definition parameters,smart container configuration parameters, suggested task order, and thelike) for performing the task that is passed along to other shipmentconfiguration system modules for processing. In embodiments, the jobparsing system may rely on the intelligence service 13004 forsuggestions of such requirements, including combinations of tasks thatwhen optionally adapted may satisfy the job requirement.

In embodiments, the job parsing system may include and/or interface withthe analysis modules/governance libraries of the intelligence service13004 of the system 13000. The job parsing system may leverage thegovernance-based analyses by providing portions of the candidate smartcontainer automation portions of the job content (e.g., terms and thelike) for processing. The intelligence service 13004 may, in response tothe provided portion of job content, provide and/or indicate one or moreof safety standards and/or one or more of operational standards to beapplied during preparation of the order execution plan by the jobconfiguration system 13018.

In embodiments, the job parsing system may include a job requirementsmodule that produces a set of freight storage and/or transportationservice order instance-specific requirements for use when the jobconfiguration system 13018 defines smart container tasks, configuresfleet resources, defines workflows, simulates workflows, generates anorder execution plan, and/or the like. In embodiments, the set offreight storage and/or transportation service order instance-specificrequirements may be determined based on at least one or more of: thecandidate portions of the job content that indicate smart containerautomation (e.g., terms that indicate a smart container task), one ormore inputs from the user interface (e.g., clarification of terms),safety and operational standards (e.g., from the governance layer), anda recommended smart container task and associated contextual information(e.g., provided by a fleet intelligence service).

In embodiments, the job parsing system may apply content filters and/orstructural filters to identify structural elements in the job contentthat may indicate one or more of tasks, sub-tasks, task ordering, taskdependencies, task requirements, and the like. In embodiments, thedetected structural elements may facilitate selection and configurationof smart container operating units by, for example, the fleetconfiguration system 13020. In an example, a structural element thatdistinguishes a set of tasks may be used by the fleet configurationsystem to avoid assigning the same smart container operating unit totasks within the set of tasks delineated by the structural element andtasks outside of the set.

In embodiments, the job parsing system may incorporate and/or utilize afreight storage and/or transportation service order configurationagent/expert system that may be constructed to facilitate developing jobdescription parsing capabilities.

In embodiments, the task definition system may organize job data intotask definitions (e.g., discrete smart container tasks or tasksperformed by smart container teams). The task definition system mayfurther coordinate other systems of the job configuration system 13018,such as a workflow simulation system to optimize the task definitions.

In embodiments, the task definition system may refine job data compiledby the job parsing system to facilitate defining discrete operations ofone or more smart container operating units in the fleet of smartcontainers in performance of a requested job. Defining tasks may bebased on information regarding smart containers, shipping containerports, container ships, railways, trucks, hyperloops, smart containertypes, smart container features, and smart container configurations thatcan perform a defined task. In embodiments, the task definition systemmay further provide information in task definitions that facilitate afleet configuration system 13020 in determining use of smart containersfor each defined task. In embodiments, the task definition system mayreference the library 13044, the intelligence service 13004, or othersystem-specific or accessible resources when making task suggestions.

As the task definition system defines the tasks of a job, the taskdefinition may be cataloged and stored for future use, such as in thelibrary 13044. In some embodiments, the task definition system may adapta task definition from a previously cataloged task definition (e.g.,adapting a task definition for a particular type of environment orcertain conditions thereof from a previously catalogued taskdefinition). In these embodiments, the task definition system maycatalogue the derivative task definition in the library 13044 withadaptation instructions. In some embodiments, a task definition that iscatalogued in the library 13044 may be associated with an alreadycataloged task definition and/or may replace an already cataloged taskdefinition, may be cataloged as a sub-task of an existing task, and thelike. In general, task definition may include associated tasks,serialized tasks, nested tasks, and the like.

Information about a job may be stored in the library 13044 for futureuse; therefore, the task definition system may access the library 13044to retrieve information about the job, smart containers, fleets, and thelike. In an exemplary embodiment of self-stacking smart containers on acontainer ship, the information accessible through the library 13044 mayinclude, for example, how to access information about the physicallayout of the container ship. The task definition system may also accessthe library 13044 to update information, such as by adding one or moretasks to a list of tasks for the self-stacking task, results fromoptimizations of task definition performed by the order executionsystem, and the like.

Optimization features of the task definition system are described belowin association with feedback from other elements of the jobconfiguration system 13018, such as the workflow simulation system andthe like.

Task definitions may be generated and provided to other elements of thejob configuration system 13018, such as the workflow definition systemand a fleet configuration system proxy. In embodiments, the fleet thatmay provide the task definitions (which may include route definitionsand other suitable information) to the fleet configuration system 13020.In an example, a fleet configuration system proxy may narrow down setsof candidate smart containers for performing tasks (as indicated in taskdescriptions) to a specific smart container type (and optionally aspecific smart container in the fleet) based on fleet configuration andfleet resource inventory and allocation data relevant to the requestedjob (e.g., based on geography, timing, and the like). The fleetconfiguration system proxy may process task definitions, which mayinclude smart container identification information (e.g., smartcontainer type and the like), for aligning resources of the fleet withthe relevant task information. In an example, a fleet configurationsystem proxy may generate data suitable for use by fleet operationalelements, such as a fleet resource provisioning system 13014, to performfleet resource allocation, scheduling, and the like that supports atleast a portion of the goals of a freight storage and/or transportationservice order being processed through the job configuration system13018. The fleet configuration system proxy may employ fleetconfiguration modeling to determine candidate fleet configurations thatmeet job requirements. The modeling may be useful in determining animpact on fleet resources that may then be taken into considerationduring fleet configuration functions, resource allocation, and the like.In embodiments, fleet configuration modeling may include use of systemintelligence service resources, such as machine learning, artificialintelligence, and the like when determining one or more preferred fleetconfigurations that also satisfy one or more job descriptionrequirements. The fleet configuration system 13020 is described infurther detail elsewhere in this disclosure.

In embodiments, the job configuration system 13018 may include aworkflow definition system that receives task definitions from the taskdefinition system, fleet configuration information from the fleetconfiguration system 13020, other freight storage and/or transportationservice order information that may facilitate task sequencing (e.g.,timing of deliverables and/or tasks), and generates one or more taskworkflows based thereon. In embodiments, the workflow definition systemincorporates information from the smart container fleet managementsystem to identify workflow possibilities using output from the taskdefinition, job parsing system, and real-time external data such asmaintenance management systems, ERP systems, and so forth to determinethe task workflows. In embodiments, a task workflow defines an order andmanner in which tasks are performed for a project/job. In embodiments,the workflow definition system may apply job descriptive information toa set of task definitions and fleet configuration data to produce one ormore workflows to perform one or more activities of the job. As anexample, a workflow may cover an activity such as last-mile 3D printingof an athletic shoe within the container. The tasks defined for thisactivity may be collected into a workflow or portion thereof, ordered toensure proper compliance with the job requirements, and published as aset of requirements to perform the activity/workflow. A job workflowdefinition may include information descriptive of quantities and typesof smart containers, 3D printers, robots, tools and/or end effectors,and the like that may be provided by the fleet configuration system13020 for one or more tasks being ordered by the workflow definitionsystem. In embodiments, this portion of the workflow definition may beutilized by other modules of the job configuration system 13018 (e.g.,order execution system 13022) to, for example, identify and determinerequired configurations of one or more smart containers, and the like tobe readied ahead of performing a task in the workflow (e.g., ensuringthat a smart container is (re)configured with a configuration thatenables performing a task prior to performing the task that is definedin the workflow). Other information produced in an order execution planmay include sequence of tasks (e.g., as produced by a workflow system),which may further identify a sequence of smart containers required toperform the tasks.

A workflow definition system may utilize resources of the smartcontainer configuration library when defining workflows. Workflowdefinition parameters, such as how to determine minimum time betweentasks, inter-task coordination, task classification, workflow scope, andthe like may be available in the library 13044, and/or in informationretrieved from a freight storage and/or transportation service order.These and other parameters may include job-specific variables that canbe set to default values, but adjusted by, for example, the workflowdefinition system to meet job-specific needs. An example of use of smartcontainer configuration library information to develop job workflowdefinitions may include a cargo unloading task (e.g., by a set ofrobotic arms attached to the smart container or by robots embeddedwithin the container), followed by a self-stacking storage task (e.g.,by an on-container rail and/or lift system). Useful information that aworkflow definition system may utilize from a smart containerconfiguration library may include template, preconfigured or defaultworkflows, such as workflows developed for a previous execution of thejob. A workflow definition system may determine which, if any, workflowin the library 13044 (base workflow) is suitable for use in the currentjob workflow definition instance; determine adjustments to the retrievedworkflow; produce an instance-specific job workflow that may includeadditional tasks not found in the base workflow and/or excludeunnecessary tasks found in the base workflow, and the like.

Other examples of smart container configuration library information thatmay be useful for developing job workflow definitions includeavailability of sensor detection packages. These sensor detectionpackages may indicate a preferred sequence of sensing tasks andtherefore may impact workflows of such tasks. These and relatedreconfigured sensor and detection packages may combine sensor selection,sensing, information collection, preprocessing, routing, consolidation,processing, and the like. These sensor and detection packages may beincluded in a fleet configuration process, such as being included in anorder execution plan for use by the order execution, monitoring, andreporting system 13022. In embodiments, use thereof is indicated asserving a range of monitoring activities and the like.

A job workflow definition system may examine task to task dependency(e.g., performing a second task of unloading cargo is dependent oncompleting a first task of transporting the cargo to a destinationcontainer terminal) to identify potential workflow independence anddependence for, among other things, configuring an order execution planthat may include parallelized use of fleet resources, such as teams andthe like.

Features of an intelligence service, such as digital twin capabilitiesand the like, may also be beneficially applied to simulate and validateworkflows, such as with a workflow simulation system of the jobconfiguration system 13018. The workflow simulation system may performsimulations of portions of a job configuration, such as those portionsorganized into job workflows by the workflow definition system. In anexample of workflow simulation, a set of tasks defined by the taskdefinition system and organized into a portion of a job workflow may bemodeled using functional equivalents for smart containers, tasks,workflows and the like, such as smart container digital twins 13504,container ship digital twins, shipping container port digital twins,railway digital twins, truck digital twins, environment digital twins,task digital twins, workflow digital twins, team digital twins, fleetdigital twins, and the like. These digital twins may be retrieved fromthe library 13044 and executed by a processor to simulate the set oftasks, such as to validate the defined tasks. In embodiments, the fleetintelligence system may be utilized for providing at least a portion ofthese workflow simulations, such as by applying workflow definitions andtask definitions to one or more workflow models and/or digital twinsoperating in an artificial intelligence environment machine learningenvironment.

The workflow simulation system may also generate feedback fromsimulating workflows defined by the workflow definition system that maybe useful in improving a workflow definition, a task definition, a smartcontainer selection and the like.

The workflow simulation system may establish or otherwise accesscriteria for determining if a workflow meets the criteria, such astimely and successfully completing a task, job, and the like. Byapplying these criteria for measuring outcomes of workflow simulations,the workflow simulation system may validate one or more workflowoptions, smart container options passed along to the workflow definitionsystem, fleet configuration options, and the like before providingfeedback to, for example, the task definition system, the job parsingsystem and the like. Options that do not meet the criteria (e.g.,consumes an excess of resources, results in wear down of a smartcontainer, fails to meet a schedule, results in a high percentage ofdamaged cargo, and the like) may be marked as such for improving jobconfiguration functions, such as structuring tasks into workflows andthe like.

Further, a workflow simulation system may leverage the systemintelligence service. In embodiments, the system intelligence servicemay provide access to and operation of instances of fleet digital twinmodules that may provide critical understanding of fleet-based impactson workflow definition for performing a requested job. In embodiments, alogistics digital twin of the fleet intelligence system may provideuseful workflow simulation information through operation of modeling ofshipments and costs of smart containers, personnel, support equipmentand the like for smart container fleet services. This modeling of fleetlogistics may reveal that a first local fleet that will soon becomeavailable (perhaps after the preferred start date of a requested job)may complete the job at a lower cost than using a second currentlyavailable fleet. In embodiments, a fleet digital twin may facilitateidentifying smart container operational assets that are available duringthe scheduled job by modeling fleet operations, such as smart containermaintenance requirements for smart containers during the preferred orderexecution time. In embodiments, a task digital twin capability of thefleet intelligence system may facilitate modeling of smart containercargo configurations, such as when a smart container cargo isreconfigured during a job. A task digital twin capability of the fleetintelligence system may further benefit workflow definition claritythrough workflow simulation by applying a virtual set of preconfiguredsmart container digital twins 13504 to perform a candidate workflow, orportion thereof, that is optionally being defined. In embodiments, ateam digital twin capability of a fleet intelligence system may benefita workflow simulation system of the job configuration system 13018 byusing, for example, preconfigured smart container teams to operate andvalidate candidate workflows prepared by the workflow definition system.

In embodiments, a result of workflow simulation may include one or moredata structures that are suitable for use in an order execution plan.

In addition to task definitions, smart container definitions, workflowdefinitions, fleet configuration parameters, and the like, an orderexecution plan may identify contracts for the job, such as smartcontracts that may be constructed/configured by or in association withthe job configuration system 13018, delivery times for job resources(e.g., fleets of smart containers), a schedule of deliverables, and thelike.

In embodiments, the fleet configuration system 13020 configuresresources of a fleet for a job based on the task definitions, workflowdefinitions, or the like. The fleet configuration system 13020 maydetermine the fleet configuration based on other considerations, such ascost, mode(s) of transportation (e.g., container ship, railway, truck,container hyperloop, and/or self-driving container), environmentalconditions, time constraints, available inventory of smart containersand/or parts, and/or the like. The fleet configuration system 13020 mayoperate cooperatively with a job configuration system 13018, such aswhen tasks are to be organized into workflows. Job workflows may beimpacted by availability of each type of smart container, so a jobconfiguration system 13018 may leverage the fleet configuration system13020 when determining candidate job workflows.

In embodiments, fleet configuration for a requested job may includeconfiguring fleet resources into a smart container team that is assignedto a specific task and/or project (noting that a smart container or ateam of smart containers may be assigned multiple tasks and/orprojects). Each smart container team may include one or more smartcontainer operating units, which may comprise any one or more of smartcontainers, robots, humans, modes of transportation, machinery (e.g., 3Dprinters), tools, and the like. Further, a configured smart containerteam may be job-specific and team membership may be transient for anygiven smart container operating unit. As an example, a robot configuredto perform cargo loading and/or unloading operations may be assigned toa first smart container team for only the duration of time during whichcargo loading and/or unloading operations are being performed by thefirst smart container team. The same robot may also be assigned to asecond smart container team for only the duration of time during whichsecond smart container team cargo unloading and/or loading is beingperformed. Time-sharing of fleet resources, such as a robots, containerships, trucks, railways, reach stackers, forklifts, cranes, or the likecan be communicated to a shipment configuration system from the fleetconfiguration system 13020, for example, so that workflows being definedby the shipment configuration system can consider availability of thecargo loading/unloading robot for each of the smart container teams. Inembodiments, any given smart container or group of smart containers maybe assigned to multiple teams spread across multiple jobs by the fleetconfiguration system 13020 using a smart container-specific time-sharingapproach or other resource utilization optimization technique. In anexample, a fleet configuration system 13020 may use a multi-dimensionalsmart container utilization planning system that allocates each smartcontainer in a fleet to one job during a unit of time, such as a day,hour, or fraction thereof, allowing each instance of a shipmentconfiguration system to request use of the smart container for aspecific period of time. The fleet configuration system 13020 mayrespond to the request with smart container fleet configurationdescriptions that inform job workflow definitions and the like.

In some embodiments, fleet configuration for a requested job may includeallocating smart container support resources, such as edge devices,charging capabilities and/or charging stations, local data storagecapabilities, container storage facilities, spare parts, humantechnicians, and the like.

In embodiments, the fleet configuration system 13020 may leveragelibraries to determine the fleet configurations. In these embodiments,the fleet configuration system 13020 may determine team configurationsfor defined tasks or projects using a library 13044 that definesdifferent configurations to perform certain tasks, whereby a lookuptable or other association is used to determine the team configurationsfor given a set of tasks. In embodiments, the library 13044 may includeattributes of different smart container types. As an example, anattribute of a smart container may indicate size or volume of the smartcontainer. In embodiments, the fleet configuration system 13020 mayfilter the types of smart containers that may perform a task based onthe attributes and one or more freight storage and/or transportationservice order parameters identified by the job parsing system (andoptionally configured into a task definition). When a task or joboperation requires (e.g., based on data generated by the job parsingsystem, an existing order execution plan, a freight storage and/ortransportation service order, and the like) access through a tunnel thatis smaller than the size smart container available, the fleetconfiguration system 13020 would not include the smart container;instead it would attempt to identify a different smart container and/orsmart container type/configuration that could meet the tunnel sizerequirements. In embodiments, a fleet configuration system 13020 mayreference combinations of smart container sizes and/or types and thelike to fit requirements of a defined task. Further, the fleetconfiguration system 13020 may suggest two smart containers to perform atask when one may not meet other requirements of the task. Inembodiments, the fleet configuration system 13020 may deliver to the jobconfiguration system 13018 fleet definitions that include a plurality ofsmart containers, smart container types, smart container configurations,and the like. A general goal of a fleet configuration system 13020 mayinclude generating fleet configuration(s) that require the fewest smartcontainers and/or smart container types for proper execution of aportion of the order. However, the fleet configuration system 13020 maywork cooperatively with the task definition system to generate atask-specific fleet configuration that includes more than one smartcontainer type/configuration combination, thereby allowing otherelements of the smart container system 13000 to efficiently manageexecution of a requested job. Such a fleet configuration may indicate apreferred smart container and/or smart container combination for meetinga goal, such as efficient use of smart containers and the like thatother elements of the shipment configuration system (e.g., a jobworkflow generation system) may consider when configuring, for example,a plurality of defined smart container tasks into a job workflow.Therefore, a fleet configuration may include primary, secondary, andtertiary smart container indications for performing a task.Alternatively, a fleet configuration for a freight storage and/ortransportation service order may identify a plurality of smartcontainers, each assigned utilization weights based on criteria, such asefficient job completion, profitability, fleet smart container usepreferences and the like.

In embodiments, the fleet configuration system 13020 may reference aninventory data store to determine the available smart containers and/ormodules (e.g., physical modules and/or software modules) to configure asmart container, locations of those smart containers and/or parts,statuses of the parts (e.g., whether maintenance is due or needed foravailable smart containers or parts), and the like. In this way, thefleet configuration for a job, task, team or the like may be determinedby the available inventory of smart containers, modules, supportequipment, and/or spare parts. Further, a fleet maintenance managementsystem as described herein may track aspects of smart container statusthat may be added to and/or be supplemental to the inventory data store,such as which smart containers are being reserved from use for criticalmaintenance, which smart containers can be deployed, but with diminishedcapability due to service and/or maintenance or other concerns, statusof spare parts, or other service activities (e.g., due date, currentlocation, anticipated installation, and the like). Therefore, the fleetconfiguration system 13020 may reference and/or be informed by the fleetmaintenance management system about fleet resource maintenance knowledgethat may be job-impacting. Additionally, or alternatively, the fleetconfiguration system 13020 may request a fleet configuration from theintelligence service 13004, where artificial intelligence modules 13404may receive a set of parameters, including task definitions, workflowdefinitions, budget, environment definition, job timeline, or the likeas input, evaluate a plurality of candidate fleet configurations anddetermine a target fleet configuration that can perform the job. Inembodiments, a human can define or redefine any portion of a fleetconfiguration via a human interface of the fleet configuration system.

In embodiments, the job and fleet configurations may be fed to a digitaltwin system, whereby the digital twin system may perform a simulation ofthe job given the job and fleet configurations. The job configurationsystem 13018 and/or the fleet configuration system 13020 may iterativelyredefine the job configuration and the fleet configuration to optimize(or substantially optimize) one or more parameters, such as a jobtimeline, overall cost, smart container downtime, maintenance-relateddowntime, shipping costs, or the like. Once the job configuration system13018 and the fleet configuration system 13020 have determined the taskand workflow definitions, as well as the fleet configurations, the smartcontainer fleet management system may output the order execution plancorresponding to the freight storage and/or transportation serviceorder.

In embodiments, the fleet configuration system 13020 may leveragedigital twins when configuring fleet resources. Use of digital twinswith fleet configuration may include identifying and/or defining one ormore digital twins of one or more smart containers based on informationin the task definition. Fleet configuration may include identifyingconfiguration and/or operation of a smart container so that a smartcontainer can perform the route and/or task or a portion thereof. Suchsmart container task configuration instructions may be generated throughthe use of a digital twin for one or more of a set of candidate smartcontainers for performing a task. In an illustrative example, a smartcontainer may be associated with a plurality ofconfiguration/operational data structures for configuring the smartcontainer to perform routines, actions, routes, tasks and the like. Thefleet configuration system 13020 may identify or otherwise be providedwith one or more candidate smart container configuration data structures(e.g., from the library 13044) for use to perform a task. A portion ofsuch a candidate configuration data structure may include a rate ofmovement for moving up a ramp onto a container ship. The requested jobrequirements may explicitly or implicitly indicate that a movement rateis different than the value in the candidate configuration datastructure. In embodiments, the fleet configuration system may make anyadjustments to the candidate configuration data structure (e.g.,reducing movement rate), apply it to an instantiation of a digital twinof the candidate smart container, observe and/or evaluate the execution(e.g., simulation) of the digital twin with the adjusted configurationdata structure, and store it in the library 13044 and the like. Thenewly stored configuration data structure may be cataloged based on thefreight storage and/or transportation service order and/or otherparameters of the requested job, task, and the like to make forefficient access in the future.

A smart container configuration library may include job information,smart container information, fleet information, task definitionrules/metadata that may be useful to determine how to define smartcontainer tasks, workflow configuration rules and/or techniques, priorfreight storage and/or transportation service order results fromapplication of the shipment configuration system (e.g., prior orderexecution plans), and the like. The library 13044 may be accessed and/orupdated by functions of the job operations system. Illustrative examplesof the library 13044 are described herein variously in conjunction withjob operations system functions and features, such as job configurationand the like. As an example, the smart container configuration librarymay include specific reference to configurations of smart containersthat may be utilized during fleet configuration, order execution and thelike. In this example, the smart container configuration library mayhave references to smart container configuration data sets (e.g., datathat when uploaded to a smart container may enable the smart containerto perform a function, such as 3D printing, in-container packaging, andthe like). Further, the library may provide a cross-reference of smartcontainer configurations with other smart container-related information,such as base model, version, required features, and the like that may berequired for successful deployment of a smart container configured witha given configuration. Yet further, the library may suggest alternativesto certain combinations of smart containers and configurations, such asindicating that a newer version of a smart container model may includebuilt-in capabilities provided by a specific configuration. Therefore,the fleet configuration system may have greater flexibility in decidingwhich smart containers to deploy for different jobs. References are madeherein to the library 13044, using contextual modifiers, such as smartcontainer configuration library and the like. These contextual modifiersmay suggest one or more portions and/or instance of the library 13044for illustrative purposes only.

In embodiments, optimization features of the task definition system aredescribed below in association with feedback from other elements of thejob configuration system 13018, such as the workflow simulation systemand the like.

In some embodiments, the fleet operations system and the fleetintelligence system perform a feedback for order execution-timeiteration of configuration activities, such as for adapting andexecuting instances of an order execution plan. In these embodiments,feedback within a job configuration system 13018 facilitates iteratingconfiguration activities when producing components of an order executionplan, such as task definitions and workflow definitions. As describedfor these embodiments, the intelligence service 13004 may be used for atleast these iterations. However, it is envisioned that the resources ofthe intelligence service 13004 may also or, in addition, be used forenhancing execution of an order execution plan.

In embodiments, the order execution system 13022 of the fleet managementsystem 13002 may receive order execution plans from the jobconfiguration system 13018 responsive to, for example, a freight storageand/or transportation service order. The order execution system 13022may facilitate performance of an order execution plan by steppingthrough the plan, activating, and monitoring smart container units andother fleet resources, and providing feedback, optionally real-timefeedback based on, for example, smart container unit monitoring data.This feedback may be processed by, for example, artificial intelligencecapabilities of the intelligence service 13004 for determiningadjustments to an order execution plan, such as task definitions and thelike. When the feedback and adjustments are done in real-time or nearreal-time (e.g., before an upcoming order execution activity, such as astep in a workflow), functions of the job configuration system 13018 maybe iterated to amend an existing order execution plan, such as aninstance of a plan that is currently being executed by the orderexecution system 13022.

An artificial intelligence system of the intelligence service 13004 mayperform simulations and use the results of the simulation as one or moreas input to the job configuration system 13018 for updatingcorresponding task definitions. In embodiments, the fleet intelligencesystem may send an alert to the fleet management system 13002 regardingthe need for adapting this task definition that may be used by thesystem to update, for example, preconfigured task definitions stored inthe smart container task library 13044 and the like. Such an alert maybe used by the fleet operations system to coordinate with the orderexecution system 13022 so that pending tasks are not executed beforebeing refreshed in the order execution plan. In embodiments, the jobconfiguration system 13018 may release only portions of the orderexecution plan to the order execution system 13022 so that unreleasedportions can be adapted, thereby mitigating impacts on the orderexecution system, such as requiring work to be halted, delayed, orotherwise impaired while updates to the execution plan are made.

While the examples for job configuration and the like presented hereingenerally consider a single job being configured by the jobconfiguration system 13018, there may be many jobs being configuredconcurrently. The methods and systems for real-time or near real-timefeedback described herein may apply to any instance of job configurationactivity being performed so that feedback on task definition of a firstjob may benefit task definition of a second job, while maintainingnecessary job-isolation requirements (e.g., job identifying data may beobfuscated) to support concurrently processing freight storage and/ortransportation service orders from different entities.

In embodiments, capturing data representative of completion of arequested job may include extracting such data from a job completiondata set. This job completion data set may be constructed to facilitateidentifying information that may be useful for learning andoptimization. In an example, the job completion data set may designate,such as by use of metadata tags, logical and/or physical separation, orother indicia data that represents exceptions or large variants fromexpectation. In an example, at job completion, a percentage of damagedpackages carried by a smart container may exceed an expected and/oracceptable number. This excessive count of damaged packages may beflagged as candidate information for learning and optimization feedbackto be extracted and sent to the intelligence service 13004. Inembodiments, an order execution plan may be configured with indicatorsof types of data to be collected and used for learning and optimizationfeedback. The intelligence service 13004 may recommend to the jobconfiguration system 13018 the types of data to be so indicated based onother factors known to the fleet intelligence system, such as inquiriesmade by smart container design engineering teams and the like. Inembodiments, learning and optimization feedback may be used by the fleetintelligence service to perform, among other things, optimization ofartificial intelligence service (e.g., recommending smart containerteams, smart container types, workflows, and the like). Referring todescriptions herein, preconfigured tasks, smart containerconfigurations, team configurations, and the like may be retrieved fromthe library 13044. When these preconfigured aspects of an orderexecution plan are executed, data representative of the performancethereof may be flagged for use as learning and optimization feedback tocontinuously improve these preconfigured aspects. An outcome of use ofthis data includes field condition-adapted preconfigured tasks that mayperform better in the real world. Another outcome of use of this dataincludes improved digital twins and machine learning models.

In embodiments, a job description to be parsed may include relevant jobdescriptive details, goals, objectives, requirements, preferences, andthe like and as may be described elsewhere herein. While not allpertinent job information may be included within the request, one ormore links to ancillary job description data may be included. Ancillaryjob data may be stored remote from a freight storage and/ortransportation service order data set (e.g., may be accessed through anInternet URL of the job description). Optionally, ancillary job data maybe stored in data structures that are accessible to the smart containersystem 13000, such as in a fleet library 13044, requestor-specificstorage, and the like. Ancillary job data may include formal standards(e.g., local disturbance regulations, safety (OSHA), electrical (NEC),quality, and the like), permitting requirements (e.g., forms, steps,timing, dependencies on other tasks, and the like), legal requirements(e.g., customs requirements, relevant laws, and the like) details of thejob, shipper standards (e.g., an acceptable percentage of damagedcargo), industry norms (e.g., work hours, material selection, templates,and the like), approved vendors (e.g., from whom supplies and otherconsumables are to be acquired), references to preconfigured tasks, userinterface templates/menus/screens for each aspect of a job (e.g., how auser can request status, observe activity, change a job requirement,respond to an inquiry, and the like) and the like. The freight storageand/or transportation service order data 13096 and, if indicated, theancillary job data are processed by a task definition ingestion facilitythat works cooperatively with a job parsing system to generate jobinstance-specific content. This job instance-specific content mayinclude, among other things, initial sequence timing as may be definedin the input data (e.g., “do task A before task B”) and/or derivedtherefrom (e.g., securing cargo in the container with steel strappingnecessarily must occur after the cargo is loaded into the container).The job parsing system may interact with the data processing system13024 when converting job description data to utilize informationderived from a smart container fleet management system accessiblelibrary, such as job and fleet library 13044. The ingestion facility maystore some job description content directly into the job instancestorage, such as job identification information, links to internalancillary data and the like.

In embodiments, one or more human interactive capabilities forfacilitating job parsing and task definition may include knowledge-basedsystems (e.g., AI-based and the like) that may interact with a human(e.g., via text input, conversation-bot, haptic-input, and the like) togather information for preformatting, organizing, and vetting job andtask data. These interactions may be in lieu of or supplemental toreceiving a job description.

The job parsing system may use job descriptive information produced byor passed through the ingestion facility to construct job instancecontent suitable for task definition. The job parsing system may use theinformation provided by the ingestion facility to query content in thelibrary 13044 (e.g., via the data processing facility 13024 asoptionally depicted). Content in the library that may be useful orinformative of task definition may include job syntax (e.g., terms thatare relevant to a given job, job type, set of tasks, smart containertypes, smart container capabilities (e.g., by type, size, cost,availability, etc.), keyword-to-task cross reference, workflowdefinition rules, order execution plan format/content/structure.Further, the library may include templates for various taskdefinition-related activities, such as exemplary smart containerconfigurations (e.g., based on task keyword and the like), exemplaryteam configurations (e.g., for performing certain types or classes oftasks), task definitions, workflows and workflow definitions, exemplaryorder execution plan(s) and the like.

A keyword-based task lookup module may retrieve information in the jobinstance storage, such as task-oriented keywords and the like and applythose to the library 13044 to potentially identify preconfigured ortemplated tasks or portions thereof. As an example, a job descriptionmay include keywords, such as “submerged” and the like that may suggesta need for smart containers that are configured to travel underwater.When such keywords are combined with an action “submerged 3D printing”,the keyword-based task lookup module may identify smart container typesthat perform 3D printing and can travel underwater. If a descriptor of atask in the library aligns with one or more job description keywords,the task may be considered a candidate task for the job.

In embodiments, a task definer module may process candidate tasksprovided by the task lookup module as well as information in the jobinstance storage to form definitions for tasks to be performed by one ormore smart containers. Defining tasks may include tasks that arepredefined by standards, laws (e.g., customs), and the like. Each taskdefinition may include information useful for identifying a smartcontainer type for performing the task.

In embodiments, the task definition system may process task data derivedfrom a freight storage and/or transportation service order (e.g., asprovided by the freight storage and/or transportation service orderparser) in the context of smart container types by identifyingcharacteristics of smart container types that align with the task data.In example embodiments, the task definition system may determine thattask data indicates a characteristic of a smart container for shippingmay include traveling in an arctic environment. In this example, thetask definition system may generate a task definition for the shippingtask that includes at least a requirement for smart container selectionbased on this characteristic. In these example embodiments, the taskdefinition may further include a required degree of tolerance to coldtemperatures. The task definition system may further determine thatcharacteristics of one or more smart containers (e.g., based on taskinformation derived from the freight storage and/or transportationservice order) that may not be suitable for incorporation in a singlesmart container/smart container type. This determination may be basedon, for example, smart container characteristics and type data that isaccessible in the library 13044. In such an example, the task definitionsystem may define multiple tasks, each with smart containercharacteristics that are consistent with smart container characteristicinformation in the library 13044. In embodiments, the task definitionsystem may define a task with multiple potentially incompatible smartcontainer characteristics, optionally along with an indication of one ormore portions of the task that require each type of the multipleincompatible smart container characteristics that a fleet configurationsystem 13020 may use when configuring fleet resources, such as smartcontainers and the like. In embodiments, a task definition may includeone or more suggestions for types of smart containers for performing thetask, such as based on alignment of task requirements (e.g., derivedfrom task information of a freight storage and/or transportation serviceorder), smart container characteristics, and smart container types thatmay be available in the library 13044. As will be explained below, afleet configuration system 13020 may evaluate a task definition,including any suggested smart container types. Other exemplary data thatmay be communicated when defining a task may include task sequencedependencies that may be suitable for defining a workflow that includesthe defined task. As an example, a container self-cleaning task may berequired to be performed after an unloading task. Such a dependency maybe documented and relied upon by a workflow definition system. The taskdefiner module may save a defined task into the job instance storagewhere it may be cross referenced to job descriptive data (e.g., keywordsand the like) so that future detections of the cross-referenced keywordscan quickly result in a suitable task definition.

In embodiments, a fleet configuration system 13020 provides specificsoftware, hardware, and smart container configuration requirements forcompletion of an order execution plan. In this example construction, afleet configuration proxy module may be constructed to receive taskdefinitions from a job configuration system 13018. The fleetconfiguration proxy module may be instantiated in association withprocessing of a freight storage and/or transportation service order bythe job configuration system 13018 to facilitate access to and use offleet configuration system 13020 resources and systems. This and otherinstantiations of the fleet configuration proxy module are furtherdescribed in association with the job configuration system 13018 herein.The fleet configuration proxy module may process task definitions andforward them to fleet resource identification systems, such as a fleetsmart container operating unit identification and a fleet non-smartcontainer operating unit identification system. Each of theseidentification systems may process the task definition data providedthrough the fleet configuration proxy, separating operational data fromfleet resource data. A task definition may describe a set of fleetresources required to perform the task, such as types of smart containeroperating units, support resources (e.g., power systems, robots, cranes,communication systems, and the like). The smart container operating unittype identification system may provide job-specific smart containeroperating unit demand data to the fleet configuration scheduler. Thejob-specific smart container operating unit demand data may identifytypes and quantities of smart containers, specific smart containeroperating units (e.g., by unique identifier), smart container operatingunit capabilities, and the like.

In some embodiments, a fleet configuration scheduler may respond to afreight storage and/or transportation service order by allocating fleetresources to meet the freight storage and/or transportation serviceorder needs. These needs may be preprocessed, as described herein by ajob configuration system 13018 and specifically by the task definitionsystem to facilitate fleet configuration, allocation, and scheduling.The fleet configuration scheduler processes inputs that describe fleetinventories, such as smart container operating unit inventories andtraditional container operating unit inventories to identify candidateinventory elements for satisfying a freight storage and/ortransportation service order. These inventories may be adjusted based onexisting allocations of smart container operating units and traditionalcontainer operating units. As an example, all smart containers of a typeidentified in the smart container operating unit job-specific demanddata may be allocated throughout a duration of time within which arequested job is constrained to be performed. The fleet configurationscheduler (e.g., with support from other system resources such asintelligence service 13004, resource provisioning system 13014 and thelike) may allocate, based on conditions in the freight storage and/ortransportation service order and smart container type, equivalence dataavailable to the fleet configuration scheduler, a smart container forthe activities requested. To accomplish this allocation, an intelligenceservice 13004 may be provided with information descriptive of thefunctionality to be provided by the smart container indicated in thejob-specific demand data and information descriptive of the tasks and/oractivities required to be performed by the smart container. Othercontext, such as differences in specifications for performing tasks by aproperly configured smart container, may also be available to theintelligence service 13004. Through use of artificial intelligence,which may include determining an impact on an overall freight storageand/or transportation service order based on use of the two differentsmart container types, the intelligence service 13004 may provide smartcontainer substitution guidance to the fleet configuration scheduler.This guidance may result in allocation of a smart container andnecessary configuration data/features for use when executing an orderexecution plan that corresponds to the freight storage and/ortransportation service order that prompted this fleet configurationscheduling activity.

In embodiments, a task definition may include recommendations for one ormore types of smart containers (e.g., based on alignment of, forexample, task requirements, smart container characteristics, and smartcontainer types), and a preferred type may be designated in the taskdefinition.

The fleet configuration scheduler may rely on other fleet systems, suchas a resource provisioning module 13014 that may contribute to and/ordetermine provisioning of fleet and third-party resources and supplies.

The intelligence service 13004, the resource provisioning module 13014and other fleet systems, including the fleet configuration scheduler mayinteract with a fleet configuration modeling system that may facilitategeneration of fleet configuration options that can be considered by thefleet configuration scheduler when configuring a fleet in response tojob configuration activities and the like. The fleet configurationmodeling system may provide simulation of fleet configurations, such asby using fleet digital twins, which may optionally be associated with adigital twin module 13420 of the intelligence service 13004.

In embodiments, the fleet configuration scheduler may rely on a fleetteam organizer module that assists in determining and/or effecting teamconfigurations. Job-specific data may identify (e.g., recommend) set(s)of smart container operating units to be configured as teams. Also,job-specific data may indicate information that may be indicative ofconfiguring teams, such as co-location of smart containers at acontainer terminal and the like. The fleet team organizer module mayconfirm and/or designate team metadata for use when configuring a fleet.The team metadata may indicate team membership and time frame for themembership (e.g., from one date to another, from a start of a task untilthe task is complete, and the like).

The fleet configuration scheduler may update fleet allocation data sets(that may be used by fleet resource allocation and/or reservationcapabilities described herein), such as the fleet smart containeroperating unit allocation data set and the fleet of non-containeroperating unit allocation data set with fleet configuration allocationinformation based on configuration(s) generated for the job-specificdemand data provided. The various inputs, including fleet configurationimpacting external data 13036 (e.g., weather, location data, trafficdata, industry standards, job-specific contextual information, and thelike) may be processed, optionally iteratively, by the fleetconfiguration scheduler to produce, among other things, fleetconfigurations that may be returned to an executing instance of a jobconfiguration system 13018 via the fleet configuration proxy module.

In embodiments, a workflow definition system may be constructed togenerate definitions of workflows for requested jobs utilizing resourcesof the smart container fleet management system. The construction of theworkflow definition system may include an ingestion module that receivesand processes task definitions that may be provided from the taskdefinition system or sourced from the library 13044, and job specificfleet configuration information that may be provided from jobconfiguration system 13018 interactions with the fleet configurationsystem 13020 (e.g., via the fleet configuration proxy module).

Ingestion of task definitions and/or fleet configuration information mayinclude aligning the fleet configuration information with one or moretask definitions. As an example of aligning tasks with fleetconfiguration information, fleet configuration information may be taggedas applying to one or more tasks in the set of task definitionsingested, such as with an identifier of the task or tasks. Other ways ofaligning task definition(s) with fleet configuration information may bebased on timing of such ingestion so that, for example, when a fleetconfiguration reference/value is received contemporaneously with a taskdefinition, the ingestion module may mark these two data items asaligned. Other ways of aligning task definition(s) with fleetconfiguration information may include one or more data values in thetask definition, which may be a data set, linked list, flat file,structured data set and the like indicating fleet configurationinformation to which the task(s) should be aligned. Fleet configurationinformation may include one more task identifiers to which the fleetconfiguration information pertains and/or should be applied whengenerating workflow definitions.

Ingestion may further include processing references (e.g., URLs,hyperlinks, external names, and the like) to workflow content in thelibrary 13044 that may be found in any of the ingested content. In anexample, a task definition may include a name of a task that is storedin the library 13044. The ingestion module may identify the name by itssyntax (e.g., a prefix may be added to a task identifier that indicatesthe task is to be retrieved from the library) and/or task definitionstructuring (e.g., a list of task names stored within a subset of thetask definition that is structured to indicate the subset of tasks areto be retrieved from the library). While the examples of ingestionherein pertain to an instance of ingestion of one or more taskdefinitions, ingestion may be performed on batches of tasks. Multipleinstances of the ingestion module may be instantiated and operatingconcurrently to process a plurality of task definitions that may beperformed. Optionally, a stream of tasks definitions may be received byingestion and each task in the stream is ingested in sequence.

One or more outcomes of processing by the ingestion module may bepresented to a set of workflow definition activities, including a taskdependency determination module that may determine dependencies amongtasks, such as which tasks need to be performed in a sequence and whichtasks can be performed independently of other tasks. The task dependencydetermination module may also determine dependency of tasks on otherfactors, such as availability of fleet resources, calendar/date/time,readiness of supply materials, and the like. Dependency on other factorsmay be identified in the task definition, such as by marking a given jobstate as a start point for the task. Further, dependency on otherfactors may be attributed to a given task definition during ingestion(e.g., based on aligning a task with a fleet configuration that sets adependency on availability of fleet resources, such as a special purposesmart container and the like).

A task grouping activity may process outcomes of the task dependencyactivity to generate groups of tasks based on a range of criteria, suchas, tasks that depend on a given task being complete may be grouped forconcurrent execution. Grouping tasks may be based on dependency on fleetresource availability, so that tasks that are dependent on a fleetresource may be grouped and performed once the resource is available.The order of performance of these grouped tasks may be based oninter-task dependency. Generally, tasks may be grouped for a range ofpurposes, such as cost savings, resource guarding, job prioritization,available order execution funds, anticipated fleet resource maintenanceneeds, earliest task start/finish time, latest task start/finish timeand the like.

A task workflow step definition activity may determine which task(s) canbe organized into each step of one or more workflows. Based oninter-task dependency (or lack thereof) multiple workflows may bedefined, each workflow including one or more workflow steps that aredefined in workflow step definition activity. Further, a workflow step,once defined, may be assigned to and/or referenced in a plurality ofworkflows. When dependencies exist, such as availability of a smartcontainer for performing a task in a workflow step, a plurality ofworkflows may themselves be made dependent. Performance of other tasksin these workflows may be concurrent even if the initial task of openingthe port must be done sequentially due to the fleet resource utilizationdependency.

In embodiments, a defined workflow step may be an adapted variant of acandidate workflow step, such as a workflow step that is retrieved fromthe library 13044. The workflow step definition activity may requestinput from other fleet resource system services, such as the dataprocessing system 13024 and/or artificial intelligence modules 13404 toadapt a candidate workflow step for use when defining one or moreworkflow steps for a given job.

Information such as workflow step dependency may be utilized by aworkflow step linking activity that may receive step linkingrecommendation(s) from the intelligence service 13004 and the like.Workflow step linking activity may generate a data structure thatindicates a sequence of performing defined workflow steps (e.g., aworkflow definition. The workflow definition may include data thatcaptures job-specific workflow information, such as workflow stepordering, workflow step performance sequence, workflow stepindependence, step-by-step links to workflow steps, workflow successcriteria, cross-workflow dependencies, and/or the like.

In embodiments, workflow definition(s) may be stored in a job instancestorage where they can be referenced as needed during job configurationand/or order execution. They may be stored in the fleet library 13044where they can be referenced by other jobs, by third parties, such asshipper and the like. They may be stored elsewhere (e.g., a cloudstorage facility) based on architectural considerations, such as beingdistributed to edge computing infrastructure resources proximal to jobdeployment sites and the like.

In embodiments, workflows may be simulated as indicated in thedescription of the job configuration system 13018. Outcomes ofsimulation may be directed to, for example, the ingestion module whereingestion operations, such as alignment of fleet configuration data withtask description data may be improved. Outcomes may also be passed to asfeedback to other components of the system to improve task definition,job configuration, fleet configuration, and/or the like.

FIG. 147 illustrates a digital twin module according to some embodimentsof the present disclosure. In embodiments, the digital twin module 13420generates a set of shipping digital twins 13302 (e.g., individual smartcontainer digital twins 13504, smart container fleet digital twins,container ship digital twins, container port digital twins, smartcontainer fleet manager or operator digital twins, shipper digitaltwins, and the like). In embodiments, the digital twin module 13420maintains a set of states of the respective shipping digital twins13302, such as using sensor data obtained from respective sensor systems13304 that monitor the shipping digital twins 13302. In embodiments, thedigital twin module 13420 may include a digital twin management system13306, a digital twin I/O system 13308, a digital twin simulation system13310, a digital twin dynamic model system 13312, and/or a digital twincontrol module 13314. In embodiments, the digital twin module 13420 mayprovide a real time sensor API that provides a set of capabilities forenabling a set of interfaces for the sensors of the respective sensorsystems 13304. In embodiments, the digital twin module 13420 may includeand/or employ other suitable APIs, brokers, connectors, bridges,gateways, hubs, ports, routers, switches, data integration systems,peer-to-peer systems, and the like to facilitate the transferring ofdata to and from the digital twin module 13420. In these embodiments,these connective components may allow an IIOT sensor or an intermediarydevice (e.g., a relay, an edge device, a switch, or the like) within asensor system 13304 to communicate data to the digital twin module 13420and/or to receive data (e.g., configuration data, control data, or thelike) from the digital twin module 13420 or another external system. Inembodiments, the digital twin module 13420 may further include a digitaltwin datastore 13316 that stores shipping digital twins 13302.

A digital twin may refer to a digital representation of one or moreshipping entities, such as an individual smart container 13026, a fleetof smart containers 13026, a shipping environment (e.g., a physicallocation along a route, a shipping container port or terminal, a smartcontainer charging station, a shipping yard, a container storagefacility, a container ship, or the like), smart container machinery, aphysical object, a device, a sensor, a human, or any combinationthereof. Non-limiting examples of physical objects include smartcontainer cargo, physical barriers along a route, raw materials,manufactured products, excavated materials, boxes, dumpsters, coolingtowers, vats, pallets, barrels, palates, bins, and many more.Non-limiting examples of devices include robots, computers, vehicles(e.g., ships, cars, trucks, trains, etc.), machinery/equipment (e.g.,forklifts, cranes, reach stackers, packaging systems, sorting systems,tractors, tillers, drills, presses, assembly lines, conveyer belts,etc.), and the like. The sensors may be any sensor devices and/or sensoraggregation devices that are found in a sensor system within anenvironment. Non-limiting examples of sensors that may be implemented ina sensor system may include temperature sensors, humidity sensors,vibration sensors, LIDAR sensors, SLAM sensors, SONAR sensors, motionsensors, chemical sensors, audio sensors, pressure sensors, weightsensors, radiation sensors, video sensors, wearable devices, relays,edge devices, crosspoint switches, and/or any other suitable sensors.Examples of different types of physical objects, devices, sensors, andenvironments are referenced throughout the disclosure.

In some embodiments, on-device sensor fusion and data storage for smartcontainers is supported, where data from multiple sensors is multiplexedat the device for storage of a fused data stream. For example, pressureand temperature data may be multiplexed into a data stream that combinespressure and temperature in a time series, such as in a byte-likestructure (where time, pressure, and temperature are bytes in a datastructure, so that pressure and temperature remain linked in time,without requiring separate processing of the streams by outsidesystems), or by adding, dividing, multiplying, subtracting, or the like,such that the fused data can be stored on the smart container. Any ofthe sensor data types described throughout this disclosure, can be fusedin this manner and stored in a local data pool, in storage, or on an IoTdevice, such as a data collector, a component of a machine, or the like.

In some embodiments, a set of digital twins may represent an entireorganization, such as shipping lines, container terminal operators,shippers, manufacturers, energy production organizations, regulatoryorganizations, governments, and the like. In these examples, the digitaltwins may include digital twins of one or more facilities of theorganization.

In embodiments, the digital twin management system 13306 generatesdigital twins. A digital twin may be comprised of (e.g., via reference)other digital twins. In this way, a discrete digital twin may becomprised of a set of other discrete digital twins. For example, adigital twin of a smart container may include digital twins of sensorson and within the smart container, digital twins of components that makeup the smart container, digital twins of smart container cargo, digitaltwins of other devices that are incorporated in or integrated with thesmart container (such as robots, 3D printers, packaging systems, orsorting systems), and the like. Taking this example one step further, adigital twin of a container terminal may include a digital twinrepresenting the layout of the terminal, including the arrangement ofphysical assets and systems in or around the terminal, digital twins ofthe shipping entities within the terminal, as well as digital twins ofstorage areas in the terminal, and the like. In this second example, thedigital twin of the container terminal may reference the embeddeddigital twins, which may then reference other digital twins embeddedwithin those digital twins.

In some embodiments, a digital twin may represent abstract entities,such as workflows and/or processes, including inputs, outputs, sequencesof steps, decision points, processing loops, and the like that make upsuch workflows and processes. For example, a digital twin may be adigital representation of a logistics workflow, a cargo loading process,a cargo unloading process, a customs process, a 3D printing process, asmart container energy charging process, or the like. In theseembodiments, the digital twin may include references to the shippingentities that are included in the workflow or process. The digital twinof the shipping process may reflect the various stages of the process.In some of these embodiments, the digital twin module 13420 may receivereal-time data from a smart container, shipping entity, and/or shippingenvironment (e.g., from a sensor system of the container port facility)in which the shipping process takes place and reflects a current (orsubstantially current) state of the process in real-time.

In embodiments, the digital representation may include a set of datastructures (e.g., classes) that collectively define a set of propertiesof a represented smart container, smart container fleet, physical object(e.g., cargo), device, sensor, or shipping environment and/or possiblebehaviors thereof. For example, the set of properties of a smartcontainer may include a type or class of smart container, the dimensionsof the smart container, the mass or weight of the smart container, thematerial(s) of the smart container, the physical properties of the smartcontainer materials(s), the surface of the smart container, the statusof the smart container, a location of the smart container, identifiersof other digital twins contained within the smart container, and/orother suitable properties.

Examples of the behaviors of a smart container may include a maximumacceleration of a smart container, a maximum speed of a smart container,ranges of motion of a smart container, a heating profile of a smartcontainer interior, a cooling profile of a smart container interior,processes that are performed by the smart container, operations that areperformed by the smart container, and the like.

The set of properties of a physical object may include a type ofphysical object, the dimensions of the physical object, the mass orweight of the physical object, the density of the physical object, thematerial(s) of the physical object, the physical properties of thematerials(s), the surface of the physical object, the status of thephysical object, a location of the physical object, identifiers of otherdigital twins contained within the physical object, and/or othersuitable properties.

Examples of a behavior of a physical object may include a state of thephysical object (e.g., a solid, liquid, or gas), a melting point of thephysical object, a density of the physical object when in a liquidstate, a viscosity of the physical object when in a liquid state, afreezing point of the physical object, a density of the physical objectwhen in a solid state, a hardness of the physical object when in a solidstate, the malleability of the physical object, the buoyancy of thephysical object, the conductivity of the physical object, a burningpoint of the physical object, the manner by which humidity affects thephysical object, the manner by which water or other liquids affect thephysical object, a terminal velocity of the physical object, and thelike.

The set of properties of a device may include a type of the device, thedimensions of the device, the mass or weight of the device, the densityof the device, the material(s) of the device, the physical properties ofthe material(s), the surface of the device, the output of the device,the status of the device, a location of the device, a trajectory of thedevice, vibration characteristics of the device, identifiers of otherdigital twins that the device is connected to and/or contains, and thelike.

Examples of the behaviors of a device may include a maximum accelerationof a device, a maximum speed of a device, ranges of motion of a device,a heating profile of a device, a cooling profile of a device, processesthat are performed by the device, operations that are performed by thedevice, and the like.

Example properties of a shipping environment may include the dimensionsof the environment, the boundaries of the environment, the temperatureof the environment, the humidity of the environment, the airflow of theenvironment, the physical objects in the environment, currents of theenvironment (if a body of water), and the like. Examples of behaviors ofan environment may include scientific laws that govern the environment,processes that are performed in the environment, rules or regulationsthat must be adhered to in the environment, and the like.

In embodiments, the properties of a digital twin may be adjusted. Forexample, the temperature of a digital twin, the humidity of a digitaltwin, the shape of a digital twin, the material of a digital twin, thedimensions of a digital twin, or any other suitable parameters may beadjusted. As the properties of the digital twin are adjusted, otherproperties may be affected as well. For example, if the temperaturewithin a smart container is increased, the pressure within the smartcontainer may increase as well, such as a pressure of a gas inaccordance with the ideal gas law. In another example, if a digital twinof a subzero shipping environment is increased to above freezingtemperatures, the properties of an embedded twin of water in a solidstate (i.e., ice) may change into a liquid state over time.

Digital twins may be represented in many different forms. Inembodiments, a digital twin may be a visual digital twin that isrendered by a computing device, such that a human user can view digitalrepresentations of a smart container fleet, individual smart containers,physical objects (e.g., cargo or the like), devices, sensors, and/orshipping environments. In embodiments, the digital twin may be renderedand output to a display device. In some of these embodiments, thedigital twin may be rendered in a graphical user interface (e.g., ascalable vector graphics (SVG) enabled user interface), such that a usermay interact with the digital twin. For example, a user may “drill down”on a particular element (e.g., a smart container) to view additionalinformation regarding the element (e.g., the state of a smart container,properties of the smart container, or the like). In some embodiments,the digital twin may be rendered and output in a virtual realitydisplay. For example, a user may view a 3D rendering of a shippingenvironment (e.g., using monitor, augmented reality headset, or virtualreality headset). While doing so, the user may view/inspect digitaltwins of smart containers, physical assets, devices, or the like in theenvironment.

In some embodiments, a data structure of the visual digital twins (i.e.,digital twins that are configured to be displayed in a 2D or 3D manner)may include surfaces (e.g., splines, meshes, polygons meshes, or thelike). In some embodiments, the surfaces may include texture data,shading information, and/or reflection data. In this way, a surface maybe displayed in a more realistic manner. In some embodiments, suchsurfaces may be rendered by a visualization engine (not shown) when thedigital twin is within a field of view and/or when existing in a largerdigital twin (e.g., a digital twin of a shipping environment). In theseembodiments, the digital twin module 13420 may render the surfaces ofdigital objects, whereby a rendered digital twin may be depicted as aset of adjoined surfaces.

In embodiments, a user may provide input that controls one or moreproperties of a digital twin via a graphical user interface. Forexample, a user may provide input that changes a property of a digitaltwin. In response, the digital twin module 13420 can calculate theeffects of the changed property and may update the digital twin and anyother digital twins affected by the change of the property.

In embodiments, a user may view processes being performed with respectto one or more digital twins (e.g., last-mile customization of a productvia a 3D printer in the smart container, cargo inspection, cargosorting, and the like). In these embodiments, a user may view the entireprocess or specific steps within a process.

In some embodiments, a shipping digital twin (and any digital twinsembedded therein) may be represented in a non-visual representation (or“data representation”). In these embodiments, a digital twin and anyembedded digital twins exist in a binary representation, but therelationships between the digital twins are maintained. For example, inembodiments, each digital twin and/or the components thereof may berepresented by a set of physical dimensions that define a shape of thedigital twin (or component thereof). Furthermore, the data structureembodying the digital twin may include a location of the digital twin.In some embodiments, the location of the digital twin may be provided ina set of coordinates. For example, a digital twin of a shippingenvironment may be defined with respect to a coordinate space (e.g., aCartesian coordinate space, a polar coordinate space, or the like). Inembodiments, embedded digital twins may be represented as a set of oneor more ordered triples (e.g., [x coordinate, y coordinate, zcoordinates] or other vector-based representations). In some of theseembodiments, each ordered triple may represent a location of a specificpoint (e.g., center point, top point, bottom point, or the like) on theshipping entity (e.g., smart container, object, device, sensor, or thelike) in relation to the environment in which the shipping entityresides. In some embodiments, a data structure of a digital twin mayinclude a vector that indicates a motion of the digital twin withrespect to the environment. For example, fluids (e.g., liquids orgasses) or solids may be represented by a vector that indicates avelocity (e.g., direction and magnitude of speed) of the entityrepresented by the digital twin. In embodiments, a vector within adigital twin may represent a microscopic subcomponent, such as aparticle within a fluid, and a digital twin may represent physicalproperties, such as displacement, velocity, acceleration, momentum,kinetic energy, vibrational characteristics, thermal properties,electromagnetic properties, and the like.

In some embodiments, a set of two or more digital twins may berepresented by a graph database that includes nodes and edges thatconnect the nodes. In some implementations, an edge may represent aspatial relationship (e.g., “abuts”, “rests upon”, “contains”, and thelike). In these embodiments, each node in the graph database representsa digital twin of an entity (e.g., a shipping entity) and may includethe data structure defining the digital twin. In these embodiments, eachedge in the graph database may represent a relationship between twoentities represented by connected nodes. In some implementations, anedge may represent a spatial relationship (e.g., “abuts”, “rests upon”,“interlocks with”, “bears”, “contains”, and the like). In embodiments,various types of data may be stored in a node or an edge. Inembodiments, a node may store property data, state data, and/or metadatarelating to a facility, system, subsystem, and/or component. Types ofproperty data and state data will differ based on the entity representedby a node. For example, a node representing a shipping robot may includeproperty data that indicates a material of the robot, the dimensions ofthe robot (or components thereof), a mass of the robot, and the like. Inthis example, the state data of the robot may include a current pose ofthe robot, a location of the robot (e.g., within a smart container or ona container ship), or the like. In embodiments, an edge may storerelationship data and metadata data relating to a relationship betweentwo nodes. Examples of relationship data may include the nature of therelationship, whether the relationship is permanent (e.g., a fixedcomponent would have a permanent relationship with the structure towhich it is attached or resting on), and the like. In embodiments, anedge may include metadata concerning the relationship between twoentities. For example, a sensor may take measurements relating to astate of a smart container, whereby one relationship between the sensorand the smart container may include “measured” and may define ameasurement type that is measured by the sensor. In this example, themetadata stored in an edge may include a list of N measurements takenand a timestamp of each respective measurement. In this way, temporaldata relating to the nature of the relationship between two entities maybe maintained, thereby allowing for an analytics engine,machine-learning engine, and/or visualization engine to leverage suchtemporal relationship data, such as by aligning disparate data sets witha series of points in time, such as to facilitate cause-and-effectanalysis used for prediction systems.

In some embodiments, a graph database may be implemented in ahierarchical manner, such that the graph database relates a set offacilities, systems, and components. For example, a digital twin of ashipping environment may include a node representing the shippingenvironment. The graph database may further include nodes representingvarious systems within the shipping environment, such as nodesrepresenting a smart container fleet, a smart container charging area, astorage area, and the like, all of which may connect to the noderepresenting the shipping environment. In this example, each of thesystems may further connect to various subsystems and/or components ofthe system. For example, the smart container system may connect to asubsystem node representing a cooling system of the smart container, asecond subsystem node representing a heating system of the smartcontainer, a third subsystem node representing the fan system of thesmart container, and one or more nodes representing a thermostat of thesmart container (or multiple thermostats). Carrying this examplefurther, the subsystem nodes and/or component nodes may connect tolower-level nodes, which may include subsystem nodes and/or componentnodes. For example, the subsystem node representing the coolingsubsystem may be connected to a component node representing an airconditioner unit. Similarly, a component node representing a thermostatdevice may connect to one or more component nodes representing varioussensors (e.g., temperature sensors, humidity sensors, and the like).

In embodiments, where a graph database is implemented, a graph databasemay relate to a single environment or may represent a larger enterprise.In the latter scenario, a company may have various shipping distributionfacilities. In these embodiments, an enterprise node representing theenterprise may connect to environment nodes of each respective facility.In this way, the digital twin module 13420 may maintain digital twinsfor multiple shipping facilities of an enterprise.

In embodiments, the digital twin module 13420 may use a graph databaseto generate a digital twin that may be rendered and displayed and/or maybe represented in a data representation. In embodiments, the digitaltwin module 13420 may receive a request to render a digital twin,whereby the request includes one or more parameters that are indicativeof a view that will be depicted. For example, the one or more parametersmay indicate a shipping environment to be depicted and the type ofrendering (e.g., “real-world view” that depicts the environment as ahuman would see it, an “infrared view” that depicts objects as afunction of their respective temperature, an “airflow view” that depictsthe airflow in a digital twin, or the like). In response, the digitaltwin module 13420 may traverse a graph database and may determine aconfiguration of the environment to be depicted based on the nodes inthe graph database that are related (either directly or through alower-level node) to the environment node of the environment and theedges that define the relationships between the related nodes. Upondetermining a configuration, the digital twin module 13420 may identifythe surfaces that are to be depicted and may render those surfaces. Thedigital twin module 13420 may then render the requested digital twin byconnecting the surfaces in accordance with the configuration. Therendered digital twin may then be output to a viewing device (e.g., VRheadset, AR headset, monitor, or the like). In some scenarios, thedigital twin module 13420 may receive real-time sensor data from asensor system of an environment and may update the visual digital twinbased on the sensor data. For example, the digital twin module 13420 mayreceive sensor data (e.g., vibration data from a vibration sensor)relating to smart container cargo. Based on the sensor data, the digitaltwin module 13420 may update the visual digital twin to indicate theapproximate vibrational characteristics of the cargo within a digitaltwin of the smart container.

In scenarios where the digital twin module 13420 is providing datarepresentations of digital twins (e.g., for dynamic modeling,simulations, machine learning), the digital twin module 13420 maytraverse a graph database and may determine a configuration of theenvironment to be depicted based on the nodes in the graph database thatare related (either directly or through a lower level node) to theenvironment node of the environment and the edges that define therelationships between the related nodes. In some scenarios, the digitaltwin module 13420 may receive real-time sensor data from a sensor systemof a shipping entity and/or environment and may apply one or moredynamic models to the digital twin based on the sensor data. In otherscenarios, a data representation of a digital twin may be used toperform simulations, as is discussed in greater detail throughout thespecification.

In some embodiments, the digital twin module 13420 may execute a digitalghost that is executed with respect to a digital twin of a shippingentity (e.g., a smart container fleet or individual smart container)and/or a shipping environment. In these embodiments, the digital ghostmay monitor one or more sensors of a sensor system of a shipping entityand/or environment to detect anomalies that may indicate a maliciousvirus, compromised sensors, or other security issues.

As discussed, the digital twin module 13420 may include a digital twinmanagement system 13306, a digital twin I/O system 13308, a digital twinsimulation system 13310, a digital twin dynamic model system 13312,and/or a digital twin control module 13314.

In embodiments, the digital twin management system 13306 generates newdigital twins, maintains/updates existing digital twins, and/or rendersdigital twins. The digital twin management system 13306 may receive userinput, uploaded data, and/or sensor data to create and maintain existingdigital twins. Upon creating a new digital twin, the digital twinmanagement system 13306 may store the digital twin in the digital twindatastore 13316. Creating, updating, and rendering digital twins arediscussed in greater detail throughout the disclosure.

In embodiments, the digital twin I/O system 13308 receives input fromvarious sources and outputs data to various recipients. In embodiments,the digital twin I/O system receives sensor data from one or more sensorsystems. In these embodiments, each sensor system may include one ormore sensors that output respective sensor data. Each sensor may beassigned an IP address or may have another suitable identifier. Eachsensor may output sensor packets that include an identifier of thesensor and the sensor data. In some embodiments, the sensor packets mayfurther include a timestamp indicating a time at which the sensor datawas collected. In some embodiments, the digital twin I/O system 13308may interface with a sensor system via the real-time sensor API. Inthese embodiments, one or more devices (e.g., sensors, aggregators, edgedevices) in the sensor system may transmit the sensor packets containingsensor data to the digital twin I/O system 13308 via the API. Thedigital twin I/O system may determine the sensor system that transmittedthe sensor packets and the cargo thereof and may provide the sensor dataand any other relevant data (e.g., time stamp, environmentidentifier/sensor system identifier, and the like) to the digital twinmanagement system 13306.

In embodiments, the digital twin I/O system 13308 may receive importeddata from one or more sources. For example, the digital twin module13420 may provide a portal for users to create and manage their digitaltwins. In these embodiments, a user may upload one or more files (e.g.,image files, LIDAR scans, blueprints, and the like) in connection with anew digital twin that is being created. In response, the digital twinI/O system 13308 may provide the imported data to the digital twinmanagement system 13306. The digital twin I/O system 13308 may receiveother suitable types of data without departing from the scope of thedisclosure.

In some embodiments, the digital twin simulation system 13310 isconfigured to execute simulations using the digital twin. For example,the digital twin simulation system 13310 may iteratively adjust one ormore parameters of a digital twin and/or one or more embedded digitaltwins. In embodiments, the digital twin simulation system 13310, foreach set of parameters, executes a simulation based on the set ofparameters and may collect the simulation outcome data resulting fromthe simulation. Put another way, the digital twin simulation system13310 may collect data relating to the properties of the digital twinand the digital twins within or containing the digital twin used duringthe simulation as well as any outcomes stemming from the simulation. Forexample, in running a simulation on a digital twin of a potential newsmart container design, the digital twin simulation system 13310 canvary the dimensions, materials, capabilities, and/or other relevantparameters and can execute simulations that output outcomes resultingfrom different combinations of the parameters. In another example, thedigital twin simulation system 13310 may simulate the vibration of cargowithin a smart container. In this example, the digital twin of the smartcontainer may include a set of operating parameters of the smartcontainer. In some embodiments, the operating parameters may be variedto evaluate the effect of the operating parameters on cargo damage. Thedigital twin simulation system 13310 is discussed in further detailthroughout the disclosure.

In embodiments, the digital twin dynamic model system 13312 isconfigured to model one or more behaviors with respect to a digital twinof an environment. In embodiments, the digital twin dynamic model system13312 may receive a request to model a certain type of behaviorregarding a shipping entity, environment, or process and may model thatbehavior using a dynamic model, the digital twin of the environment orprocess, and sensor data collected from one or more sensors that aremonitoring the environment or process. For example, an operator of asmart container fleet may wish to model the performance of the fleet todetermine whether the fleet can withstand an increase in freighttransportation service demand. In this example, the digital twin dynamicmodel system 13312 may execute a dynamic model that is configured todetermine whether an increase in demand would result in adverseconsequences (e.g., failures, downtime, or the like). The digital twindynamic model system 13312 is discussed in further detail throughout thedisclosure.

In embodiments, the intelligence service 13004 performs machine learningand artificial intelligence-related tasks on behalf of the digital twinsystem. In embodiments, the intelligence service 13004 train machinelearned models using the output of simulations executed by the digitaltwin simulation system 13310. In some of these embodiments, the outcomesof the simulations may be used to supplement training data collectedfrom real-world environments and/or processes. In embodiments, theintelligence service 13004 leverages machine learned models to makepredictions and classifications and provide decision support relating tothe real-world environments and/or processes represented by respectivedigital twins.

For example, a machine-learned prediction model may be used to predictthe cause of irregular vibrational patterns for a bearing of an engineof a smart container. In this example, the intelligence service 13004may receive vibration sensor data from one or more vibration sensorsdisposed on or near the engine and may receive maintenance data from thesmart container and may generate a feature vector based on the vibrationsensor data and the maintenance data. The intelligence service 13004 mayinput the feature vector into a machine-learned model trainedspecifically for the engine (e.g., using a combination simulation dataand real-world data of causes of irregular vibration patterns) topredict the cause of the irregular vibration patterns. In this example,the causes of the irregular vibrational patterns could be a loosebearing, a lack of bearing lubrication, a bearing that is out ofalignment, a worn bearing, the phase of the bearing may be aligned withthe phase of the engine, loose housing, loose bolt, and the like.

In embodiments, the digital twin control module 13314 controls one ormore aspects of smart containers, smart container fleets, and/or othershipping entities and environments. In embodiments, the digital twincontrol module 13314 may leverage the digital twin simulation system13310, the digital twin dynamic model system 13312, and/or theintelligence service 13004 to determine one or more controlinstructions. In embodiments, the digital twin control module 13314 mayimplement a rules-based and/or a machine-learning approach to determinethe control instructions. In response to determining a controlinstruction, the digital twin control module 13314 may output thecontrol instruction to the smart container, smart container fleet,and/or other shipping entities and environments via the digital twin I/Osystem 13308.

In embodiments, the digital twin management system 13306 may include,but is not limited to, a digital twin configuration module 13318, adigital twin update module 13320, and a digital twin visualizationmodule 13322.

In embodiments, the digital twin configuration module 13318 mayconfigure a set of new digital twins of a set of environments usinginput from users, imported data (e.g., blueprints, specifications, andthe like), image scans of the environment, 3D data from a LIDAR deviceand/or SLAM sensors, and other suitable data sources. For example, auser (e.g., a user affiliated with an organization/customer account)may, via a client application 13324 (such as via the smart containersystem 13000), provide input to create a new shipping digital twin. Indoing so, the user may upload 2D or 3D image scans and/or blueprints ofthe shipping entity and/or environment. The user may also upload 3Ddata, such as taken by a camera, a LIDAR device, an IR scanner, a set ofSLAM sensors, a radar device, an EMF scanner, or the like. In responseto the provided data, the digital twin configuration module 13318 maycreate a 3D representation of the shipping entity or environment, whichmay include any objects that were captured in the image data/detected inthe 3D data. In embodiments, the intelligence service 13004 may analyzeinput data (e.g., blueprints, image scans, 3D data) to classify rooms,pathways, equipment, and the like to assist in the generation of the 3Drepresentation. In some embodiments, the digital twin configurationmodule 13318 may map the digital twin to a 3D coordinate space (e.g., aCartesian space having x, y, and z axes).

In some embodiments, the digital twin configuration module 13318 mayoutput the 3D representation of the shipping entity and/or environmentto a graphical user interface (GUI). In some of these embodiments, auser may identify certain areas and/or objects and may provide inputrelating to the identified areas and/or objects. For example, a user maylabel specific rooms, equipment, machines, devices, sensors, and thelike. Additionally, or alternatively, the user may provide data relatingto the identified objects and/or areas. For example, in identifying asmart container, the user may provide a make/model number of the smartcontainer. In some embodiments, the digital twin configuration module13318 may obtain information from a manufacturer of a device, a piece ofequipment, or machinery. This information may include one or moreproperties and/or behaviors of the device, equipment, or machinery. Insome embodiments, the user may, via the GUI, identify locations ofsensors throughout the shipping entity and/or environment. For eachsensor, the user may provide a type of sensor and related data (e.g.,make, model, IP address, and the like). The digital twin configurationmodule 13318 may record the locations (e.g., the x, y, z coordinates ofthe sensors) in the shipping digital twin. In embodiments, the digitaltwin module 13420 may employ one or more systems that automate thepopulation of digital twins. For example, the digital twin module 13420may employ a machine vision-based classifier that classifies makes andmodels of devices, equipment, or sensors. Additionally, oralternatively, the digital twin module 13420 may iteratively pingdifferent types of known sensors to identify the presence of specifictypes of sensors that are in an environment. Each time a sensor respondsto a ping, the digital twin module 13420 may extrapolate the make andmodel of the sensor.

In some embodiments, the manufacturer may provide or make availabledigital twins of their products (e.g., sensors, devices, machinery,equipment, raw materials, and the like). In these embodiments, thedigital twin configuration module 13318 may import the digital twins ofone or more products that are identified in the environment and mayembed those digital twins in the digital twin of the environment. Inembodiments, embedding a digital twin within another digital twin mayinclude creating a relationship between the embedded digital twin withthe other digital twin. In these embodiments, the manufacturer of thedigital twin may define the behaviors and/or properties of therespective products. For example, a digital twin of a 3D printer in asmart container may define the manner by which the 3D printer operates,the inputs/outputs of the 3D printer, and the like. In this way, thedigital twin of the 3D printer may reflect the operation of the 3Dprinter given a set of inputs.

In embodiments, a user may define one or more shipping processes. Inthese embodiments, the user may define the steps in the process, themachines/devices that perform each step in the process, the inputs tothe process, and the outputs of the process.

In embodiments, the digital twin configuration module 13318 may create agraph database that defines the relationships between a set of digitaltwins. In these embodiments, the digital twin configuration module 13318may create nodes for the shipping entity and/or environment, systems,and subsystems of the shipping entity and/or environment, devices in theshipping entity and/or environment, sensors in the shipping entityand/or environment, workers that work in a shipping environment,shipping processes that are performed involving the shipping entityand/or environment, and the like. In embodiments, the digital twinconfiguration module 13318 may write the graph database representing aset of digital twins to the digital twin datastore 13316.

In embodiments, the digital twin configuration module 13318 may, foreach node, include any data relating to the entity in the noderepresenting the entity. For example, in defining a node representing acontainer ship, the digital twin configuration module 13318 may includethe dimensions, boundaries, layout, pathways, and other relevant spatialdata in the node. Furthermore, the digital twin configuration module13318 may define a coordinate space with respect to the container ship.In the case that the digital twin may be rendered, the digital twinconfiguration module 13318 may include a reference in the node to anyshapes, meshes, splines, surfaces, and the like that may be used torender the environment. In representing a system, subsystem, device, orsensor, the digital twin configuration module 13318 may create a nodefor the respective entity and may include any relevant data. Forexample, the digital twin configuration module 13318 may create a noderepresenting a shipping robot. In this example, the digital twinconfiguration module 13318 may include the dimensions, behaviors,properties, location, and/or any other suitable data relating to therobot in the node representing the robot. The digital twin configurationmodule 13318 may connect nodes of related entities with an edge, therebycreating a relationship between the entities. In doing so, the createdrelationship between the entities may define the type of relationshipcharacterized by the edge. In representing a process, the digital twinconfiguration module 13318 may create a node for the entire process ormay create a node for each step in the process. In some of theseembodiments, the digital twin configuration module 13318 may relate theprocess nodes to the nodes that represent the machinery/devices thatperform the steps in the process. In embodiments, where an edge connectsthe process step nodes to the machinery/device that performs the processstep, the edge or one of the nodes may contain information thatindicates the input to the step, the output of the step, the amount oftime the step takes, the nature of processing of inputs to produceoutputs, a set of states or modes the process can undergo, and the like.

In embodiments, the digital twin update module 13320 updates sets ofdigital twins based on a current status of one or more shipping entitiesand/or environments. In some embodiments, the digital twin update module13320 receives sensor data from a sensor system of a shipping entityand/or environment and updates the status of the digital twin of theshipping entity or environment and/or digital twins of any affectedsystems, subsystems, devices, workers, processes, or the like. Asdiscussed, the digital twin I/O system 13308 may receive the sensor datain one or more sensor packets. The digital twin I/O system 13308 mayprovide the sensor data to the digital twin update module 13320 and mayidentify the entity or environment from which the sensor packets werereceived and the sensor that provided the sensor packet. In response tothe sensor data, the digital twin update module 13320 may update a stateof one or more digital twins based on the sensor data. In some of theseembodiments, the digital twin update module 13320 may update a record(e.g., a node in a graph database) corresponding to the sensor thatprovided the sensor data to reflect the current sensor data. In somescenarios, the digital twin update module 13320 may identify certainareas within the entity or environment that are monitored by the sensorand may update a record (e.g., a node in a graph database) to reflectthe current sensor data. For example, the digital twin update module13320 may receive sensor data reflecting different vibrationalcharacteristics of a smart container and/or components. In this example,the digital twin update module 13320 may update the records representingthe vibration sensors that provided the vibration sensor data and/or therecords representing the smart container and/or the smart containercomponents to reflect the vibration sensor data. In another example, insome scenarios, workers in a shipping environment (e.g., container port,container storage facility, or the like) may be required to wearwearable devices (e.g., smart watches, smart helmets, smart shoes, orthe like). In these embodiments, the wearable devices may collect sensordata relating to the worker (e.g., location, movement, heartrate,respiration rate, body temperature, or the like) and/or the environmentsurrounding the worker and may communicate the collected sensor data tothe digital twin module 13420 (e.g., via the real-time sensor API 13326)either directly or via an aggregation device of the sensor system. Inresponse to receiving the sensor data from the wearable device of aworker, the digital twin update module 13320 may update a digital twinof a worker to reflect, for example, a location of the worker, atrajectory of the worker, a health status of the worker, or the like. Insome of these embodiments, the digital twin update module 13320 mayupdate a node representing a worker and/or an edge that connects thenode representing the environment with the collected sensor data toreflect the current status of the worker.

In some embodiments, the digital twin update module 13320 may providethe sensor data from one or more sensors to the digital twin dynamicmodel system 13312, which may model a behavior of a shipping environmentand/or one or more shipping entities to extrapolate additional statedata.

In embodiments, the digital twin visualization module 13322 receivesrequests to view a visual digital twin or a portion thereof. Inembodiments, the request may indicate the digital twin to be viewed(e.g., smart container identifier). In response, the digital twinvisualization module 13322 may determine the requested digital twin andany other digital twins implicated by the request. For example, inrequesting to view a digital twin of smart container, the digital twinvisualization module 13322 may further identify the digital twins of anyshipping entities within the smart container. In embodiments, thedigital twin visualization module 13322 may identify the spatialrelationships between the shipping entities and the smart containerbased on, for example, the relationships defined in a graph database. Inthese embodiments, the digital twin visualization module 13322 candetermine the relative location of embedded digital twins within thecontaining digital twin, relative locations of adjoining digital twins,and/or the transience of the relationship (e.g., is an object fixed to apoint or does the object move). The digital twin visualization module13322 may render the requested digital twins and any other implicateddigital twin based on the identified relationships. In some embodiments,the digital twin visualization module 13322 may, for each digital twin,determine the surfaces of the digital twin. In some embodiments, thesurfaces of a digital may be defined or referenced in a recordcorresponding to the digital twin, which may be provided by a user,determined from imported images, or defined by a manufacturer of ashipping entity. In the scenario that an object can take different posesor shapes (e.g., a shipping robot), the digital twin visualizationmodule 13322 may determine a pose or shape of the object for the digitaltwin. The digital twin visualization module 13322 may embed the digitaltwins into the requested digital twin and may output the requesteddigital twin to the client application.

In some of these embodiments, the request to view a digital twin mayfurther indicate the type of view. As discussed, in some embodiments,digital twins may be depicted in a number of different view types. Forexample, a shipping entity or environment may be viewed in a“real-world” view that depicts the environment or device as theytypically appear, in a “heat” view that depicts the environment orentity in a manner that is indicative of a temperature of theenvironment or entity, in a “vibration” view that depicts shippingentities in a manner that is indicative of vibrational characteristicsof the entities, in a “velocity” view that depicts shipping entities ina manner that is indicative of the velocity of the entities, in a“filtered” view that only displays certain types of objects orcomponents (such as objects that require attention resulting from, forexample, recognition of a fault condition, an alert, an updated report,or other factor), an augmented view that overlays data on the digitaltwin, and/or any other suitable view types.

In embodiments, digital twins may be depicted in a number of differentrole-based view types. For example, a smart container fleet may beviewed in a “manager” view that depicts the smart container in a mannersuitable for a smart container fleet manager, a container terminalenvironment may be viewed in an “operator” view that depicts thecontainer terminal in a manner that is suitable for a container terminaloperator, a “regulatory” view that depicts the facility in a manner thatis suitable for regulatory managers, a shipper may view a smartcontainer digital twin 13504 in a “shipper” view that depicts the smartcontainer in a manner suitable for a shipper, and the like. In responseto a request that indicates a view type, the digital twin visualizationmodule 13322 may retrieve the data for each digital twin thatcorresponds to the view type. For example, if a user has requested aheat view of a smart container fleet, the digital twin visualizationmodule 13322 may retrieve temperature data for that set of smartcontainers (which may include temperature measurements taken from smartcontainers, shipping environments, different smart container components,and/or temperature measurements that were extrapolated by the digitaltwin dynamic model system 13312 and/or simulated temperature data fromdigital twin simulation system 13310) as well as available temperaturedata for any other shipping entities. In this example, the digital twinvisualization module 13322 may determine colors corresponding to eachsmart container that represents a temperature fault level state (e.g.,red for alarm, orange for critical, yellow for suboptimal, and green fornormal operation). The digital twin visualization module 13322 may thenrender the digital twins of the smart containers based on the determinedcolors. It is noted that in some embodiments, the digital twin module13420 may include an analytics system (not shown) that determines themanner by which the digital twin visualization module 13322 presentsinformation to a human user. For example, the analytics system may trackoutcomes relating to human interactions with real-world environments orobjects in response to information presented in a visual digital twin.In some embodiments, the analytics system may apply cognitive models todetermine the most effective manner to display visualized information(e.g., what colors to use to denote an alarm condition, what kind ofmovements or animations bring attention to an alarm condition, or thelike) or audio information (what sounds to use to denote an alarmcondition) based on the outcome data. In some embodiments, the analyticssystem may apply cognitive models to determine the most suitable mannerto display visualized information based on the role of the user. Inembodiments, the visualization may include display of informationrelated to the visualized digital twins, including graphicalinformation, graphical information depicting physical characteristics,graphical information depicting financial characteristics, graphicalinformation depicting performance characteristics, recommendations fromintelligence service 13004, predictions from intelligence service 13004,probability of failure data, maintenance history data, time to failuredata, cost of downtime data, probability of downtime data, cost ofrepair data, cost of replacement data (e.g., replacing a smart containeror smart container component), and the like.

In embodiments, a smart container fleet manager digital twin 13328 is adigital twin configured for a manager and/or operator of a fleet ofsmart containers. In embodiments, the smart container fleet managerdigital twin 13328 may work in connection with the system 13000 toprovide simulations, optimizations, classifications, configurationand/or control, predictions, statistical summaries, and decision-supportbased on analytics, machine learning, and/or other AI and learning-typeprocessing of inputs (e.g., maritime data, traffic data, weather data,sensor data, regulatory data, and the like). In embodiments, a smartcontainer fleet manager digital twin 13328 may provide functionalityincluding, but not limited to, confirming freight storage and/ortransportation service orders, selecting smart container modes oftransportation/routes, engaging in maintenance service transactions withthird-party service providers, inspecting individual smart containers,monitoring smart container fleets, generating smart contracts,monitoring regulatory compliance, performing risk management, and otherfleet manager-related activities.

In embodiments, the types of data that may populate a smart containerfleet manager digital twin 13328 may include, but are not limited to:financial data, weather data, macroeconomic data, microeconomic data,forecast data, demand planning data, analytic results of AI and/ormachine learning modeling (e.g., financial forecasting), predictiondata, asset data, recommendation data, strategic competitive data (e.g.,news and events regarding industry trends and competitors), shippingdata, maritime data, trucking fleet data, freight data, aviation data,railway data, traffic data, social media data, survey data, and manyothers. In embodiments, the digital twin module 13420 may obtainfinancial data from, for example, publicly disclosed financialstatements, third-party reports, tax filings, public news sources, andthe like. In embodiments, macroeconomic data may be derived analyticallyfrom various financial and operational data collected by the system13000. In embodiments, the business performance metrics may be derivedanalytically, based at least in part on real-time operations data, bythe intelligence service 13004 and/or provided from other users and/ortheir respective trader digital twins.

In embodiments, a smart container fleet manager digital twin 13328 mayinclude high-level views of different states of a fleet, real-timerepresentations of the fleet, historical representations of the fleet,projected representations of the fleet (e.g., future states), real-timerepresentations of individual smart containers, historicalrepresentations of individual smart containers, projectedrepresentations of individual smart containers (e.g., future states),real-time representations of shippers, historical representations ofshippers, projected representations of shippers, real-timerepresentations of shipping lines, historical representations ofshipping lines, projected representations of shipping lines, news and/ortelevision data, economic sentiment data, social media data, charts,countdown to close information, lease terms, smart contract terms,contract terms, and many others. In embodiments, a smart container fleetmanager digital twin 13328 may allow a user to access and/or interactwith other shipping digital twins. In embodiments, a smart containerfleet manager digital twin 13328 may allow a user to access and/orinteract with a fleet of smart container digital twins 13504 and/orindividual smart container digital twins 13504. In embodiments, a smartcontainer fleet manager digital twin 13328 may allow a user to interactwith another smart container fleet manager digital twin 13328 and/or ashipper digital twin 13502. The smart container fleet manager digitaltwin 13328 may initially depict the various states at a lowergranularity level. In embodiments, a user that is viewing the smartcontainer fleet manager digital twin 13328 may select to drill down intoa selected state and view the selected state at a higher level ofgranularity. For example, the smart container fleet manager digital twin13328 may initially depict a subset of the various states of a smartcontainer fleet at a lower granularity level, including a pricing state(e.g., a visual indicator indicating pricing for smart containers). Inresponse to a selection, the smart container fleet manager digital twin13328 may provide data, analytics, summary, and/or reporting including,but not limited to, real-time, historical, aggregated, comparison,and/or forecasted pricing data (e.g., real-time, historical, simulated,and/or forecasted revenues, liabilities, and the like). In embodiments,the smart container fleet manager digital twin 13328 may initiallypresent the user (e.g., the fleet manager) with a view of variousdifferent aspects of the fleet (e.g., different indicators to indicatedifferent “health” levels of a fleet) but may allow the user to selectwhich aspects require more of his or her attention. In response to sucha selection, the smart container fleet manager digital twin 13328 mayrequest a more granular view of the selected state(s) from the system13000, which may return the requested states at the more granular level.

In embodiments, the digital twin simulation system 13310 may receive arequest to perform a simulation requested by the smart container fleetmanager digital twin 13328, where the request indicates one or moreparameters that are to be varied in one or more digital twins. Inresponse, the digital twin simulation system 13310 may return thesimulation results to the smart container fleet manager digital twin13328, which in turn outputs the results to the user via the clientdevice display. In this way, the user may be provided with variousoutcomes corresponding to different parameter configurations. Forexample, a user may request a set of simulations to be run to testdifferent fleet configurations to see how the different configurationsaffect the overall impact on profits and losses. The digital twinsimulation system 13310 may perform the simulations by varying thedifferent configurations and may output the financial forecasts for eachrespective configuration. In some embodiments, the user may select aparameter set based on the various outcomes, and iterate simulationsbased at least on the varied prior outcomes. In some embodiments, anintelligent agent may be trained to recommend and/or select a parameterset based on the respective outcomes associated with each respectiveparameter set.

In embodiments, a smart container fleet manager digital twin 13328 maybe configured to store, aggregate, merge, analyze, prepare, report, anddistribute material relating to pricing, scheduling, financialreporting, performance, maintenance, regulatory data, or other datarelated to a smart container shipping services. A smart container fleetmanager digital twin 13328 may link to, interact with, and be associatedwith external data sources, and able to upload, download, aggregateexternal data sources, including with the system 13000's internal data,and analyze such data, as described herein. Data analysis, machinelearning, AI processing, and other analysis may be coordinated betweenthe smart container fleet manager digital twin 13328 and theintelligence service 13004. This cooperation and interaction may includeassisting with seeding shipping-related data elements and domains in thedigital twin data store 13316 for use in modeling, machine learning, andAI processing to identify an optimal fleet configuration, optimalscheduling execution of freight storage and/or transportation serviceorders, or some other shipping-related metric or aspect, as well asidentification of the optimal data measurement parameters on which tobase judgement of a fleet configuration or scheduling execution success.In embodiments, the digital twin module 13420 abstracts the differentviews (or states) within the digital twin to the appropriategranularity. For instance, the digital twin module 13420 may have accessto all the sensor data collected on behalf of the 13000 as well asaccess to real-time sensor data streams. In this example, if the sensorreadings from a particular smart container are indicative of apotentially critical situation (e.g., failure state, dangerouscondition, damaged cargo, potentially illegal cargo, or the like), thenthe analytics that indicate the potentially critical situation maybecome very important to the fleet manager. Thus, the digital twinmodule 13420, when building the appropriate perspective for the fleetmanager, may include a state indicator of the smart container in thefleet manager digital twin 13328. In this way, the fleet manager candrill down into the state indicator of the smart container to view thepotentially critical situation at a greater granularity (e.g., smartcontainer machinery and an analysis of the sensor data used to identifythe situation).

In embodiments, a smart container fleet manager digital twin 13328 maybe configured to report on the performance of smart containers in thefleet. As described herein, reporting may include timing performancemetrics, financial performance metrics, physical performance metrics,cargo damage metrics, data regarding resource usage, or some other typeof reporting data. In embodiments, an intelligent agent trained by theuser may be trained to surface the most important reports to the user.For example, if the user (e.g., the fleet manager) consistently viewsand follows up on timing performance but routinely skips over reportsrelating to financial performance, the executive agent may automaticallysurface reports related to timing metrics to the user while suppressingfinancial performance data.

In embodiments, a smart container fleet manager digital twin 13328 maybe configured to monitor, store, aggregate, merge, analyze, prepare,report, and distribute material relating to other shipping entities(e.g., shippers, shipping lines, container terminals, or named entitiesof interest). In embodiments, such data may be collected by the system13000 via data aggregation, webscraping, or other techniques to searchand collect shipping entity information from sources including, but notlimited to, regulatory information, information on shipping, pressreleases, SEC or other financial reports, or some other publiclyavailable data. For example, a user wishing to monitor a certainshipping entity may request that the smart container fleet managerdigital twin 13328 provide materials relating to the certain shippingentity. In response, the system 13000 may identify a set of data sourcesthat are either publicly available or to which the fleet manager hasaccess (e.g., internal data sources, licensed 3^(rd) party data, or thelike).

In embodiments, the client application 13324, such as 13000, thatexecutes the smart container fleet manager digital twin 13328 may beconfigured with an intelligent agent that is trained on the fleetmanager's actions (which may be indicative of behaviors, and/orpreferences). In embodiments, the intelligent agent may record thefeatures relating to the actions (e.g., the circumstances relating tothe user's action) to the intelligent agent system. For example, theintelligent agent may record each time the user approves a freightstorage and/or transportation service order (which is the action) aswell as the features surrounding the approval (e.g., the type of action,the type of order, the price of the order, the shipper, the quantity ofsmart containers, route information, and the like). The intelligentagent may report the actions and features to the intelligent agentsystem that may train the intelligent agent on the manner by which theintelligent agent can undertake or recommend approval tasks and othertasks in the future. Once trained, the intelligent agent mayautomatically perform actions and/or recommend actions to the user.Furthermore, in embodiments, the intelligent agent may record outcomesrelated to the performed/recommended actions, thereby creating afeedback loop with the intelligent agent system.

In embodiments, a smart container fleet manager digital twin 13328 mayprovide an interface for a fleet manager to perform one or more fleetmanager-related workflows. For example, the smart container fleetmanager digital twin 13328 may provide an interface for a manager toperform, supervise, or monitor freight transportation order approvalworkflows, smart container maintenance workflows, logistics workflows,smart contract workflows, shipping and/or delivery workflows, regulatoryworkflows, and the like.

In another example, a user may request a filtered view of a digital twinof a process, whereby the digital twin of the process only showsshipping entities that are involved in the process. In this example, thedigital twin visualization module 13322 may retrieve a digital twin ofthe process, as well as any related digital twins (e.g., a digital twinof the environment and digital twins of any shipping entities thatimpact the process). The digital twin visualization module 13322 maythen render each of the digital twins (e.g., the environment and therelevant shipping entities) and then may perform the process on therendered digital twins. It is noted that, as a process may be performedover a period of time and may include moving items and/or parts, thedigital twin visualization module 13322 may generate a series ofsequential frames that demonstrate the process. In this scenario, themovements of the shipping entities implicated by the process may bedetermined according to the behaviors defined in the respective digitaltwins of the machines and/or devices.

As discussed, the digital twin visualization module 13322 may output therequested digital twin to a client application 13324. In someembodiments, the client application 13324 is a virtual realityapplication, whereby the requested digital twin is displayed on avirtual reality headset. In some embodiments, the client application13324 is an augmented reality application, whereby the requested digitaltwin is depicted in an AR-enabled device. In these embodiments, therequested digital twin may be filtered such that visual elements and/ortext are overlaid on the display of the AR-enabled device.

It is noted that, while a graph database is discussed, the digital twinmodule 13420 may employ other suitable data structures to storeinformation relating to a set of digital twins. In these embodiments,the data structures, and any related storage system, may be implementedsuch that the data structures provide for some degree of feedback loopsand/or recursion when representing iteration of flows.

In embodiments, a digital twin I/O system 13308 interfaces with theshipping entity and/or environment, the digital twin module 13420,and/or components thereof to provide bi-directional transfer of databetween coupled components according to some embodiments of the presentdisclosure.

In embodiments, the transferred data includes signals (e.g., requestsignals, command signals, response signals, etc.) between connectedcomponents, which may include software components, hardware components,physical devices, virtualized devices, simulated devices, combinationsthereof, and the like. The signals may define material properties (e.g.,physical quantities of temperature, pressure, humidity, density,viscosity, etc.), measured values (e.g., contemporaneous or storedvalues acquired by the device or system), device properties (e.g.,device ID or properties of the device's design specifications,materials, measurement capabilities, dimensions, absolute position,relative position, combinations thereof, and the like), set points(e.g., targets for material properties, device properties, systemproperties, combinations thereof, and the like), and/or critical points(e.g., threshold values such as minimum or maximum values for materialproperties, device properties, system properties, etc.). The signals maybe received from systems or devices that acquire (e.g., directly measureor generate) or otherwise obtain (e.g., receive, calculate, look-up,filter, etc.) the data, and may be communicated to or from the digitaltwin I/O system 13308 at predetermined times or in response to a request(e.g., polling) from the digital twin I/O system 13308. Thecommunications may occur through direct or indirect connections (e.g.,via intermediate modules within a circuit and/or intermediate devicesbetween the connected components). The values may correspond toreal-world elements or virtual elements (e.g., an input or output for adigital twin and/or a simulated element that provides data).

In embodiments, the real-world elements may be elements within ashipping entity or environment. The real-world elements may include, forexample, non-networked elements, the devices (smart or non-smart),sensors, and humans. The real-world elements may be process ornon-process equipment within the shipping entities or environments. Forexample, process equipment may include motors, cranes, reach stackers,forklifts, pumps, fans, and the like, and non-process equipment mayinclude personal protective equipment, safety equipment, emergencystations or devices (e.g., safety showers, eye wash stations, fireextinguishers, sprinkler systems, etc.), container terminal or otherfacility features (e.g., walls, floor layout, etc.), obstacles (e.g.,persons or other items within an entity or environment), and the like.

In embodiments, the virtual elements may be digital representations ofor that correspond to contemporaneously existing real-world elements.Additionally, or alternatively, the virtual elements may be digitalrepresentations of or that correspond to real-world elements that may beavailable for later addition and implementation into the entity orenvironment. The virtual elements may include, for example, simulatedelements and/or digital twins. In embodiments, the simulated elementsmay be digital representations of real-world elements that are notpresent within the shipping entity or environment. The simulatedelements may mimic desired physical properties which may be laterintegrated within the entity or environment as real-world elements(e.g., a “black box” that mimics the dimensions of a real-worldelements). The simulated elements may include digital twins of existingobjects (e.g., a single simulated element may include one or moredigital twins for existing sensors). Information related to thesimulated elements may be obtained, for example, by evaluating behaviorof corresponding real-world elements using mathematical models oralgorithms, from libraries that define information and behavior of thesimulated elements (e.g., physics libraries, chemistry libraries, or thelike).

In embodiments, the digital twin may be a digital representation of oneor more real-world elements. The digital twins are configured to mimic,copy, and/or model behaviors and responses of the real-world elements inresponse to inputs, outputs, and/or conditions of the surroundingenvironment. Data related to physical properties and responses of thereal-world elements may be obtained, for example, via user input, sensorinput, and/or physical modeling (e.g., thermodynamic models,electrodynamic models, mechanodynamic models, etc.).

Information for the digital twin may correspond to and be obtained fromthe one or more real-world elements corresponding to the digital twin.For example, in some embodiments, the digital twin may correspond to onereal-world element that is a fixed digital vibration sensor on a pieceof smart container cargo, and vibration data for the digital twin may beobtained by polling or fetching vibration data measured by the fixeddigital vibration sensor on the cargo. In a further example, the digitaltwin may correspond to a plurality of real-world elements that are eacha fixed digital vibration sensor on a smart container component, andvibration data for the digital twin may be obtained by polling orfetching vibration data measured by each of the fixed digital vibrationsensors on the plurality of real-world elements. Additionally, oralternatively, vibration data of a first digital twin may be obtained byfetching vibration data of a second digital twin that is embedded withinthe first digital twin, and vibration data for the first digital twinmay include or be derived from vibration data for the second digitaltwin. For example, the first digital twin may be a digital twin of asmart container and the second digital twin may be a digital twincorresponding to a cargo within the smart container such that thevibration data for the first digital twin is obtained from or calculatedbased on data including the vibration data for the second digital twin.

In embodiments, the digital twin module 13420 monitors properties of thereal-world elements using sensors that may be represented by a digitaltwin and/or outputs of models for one or more simulated elements. Inembodiments, the digital twin module 13420 may minimize networkcongestion while maintaining effective monitoring of processes byextending polling intervals and/or minimizing data transfer for sensorsthat correspond to affected real-world elements and performingsimulations (e.g., via the digital twin simulation system 13310) duringthe extended interval using data that was obtained from other sources(e.g., sensors that are physically proximate to or have an effect on theaffected real-world elements). Additionally, or alternatively, errorchecking may be performed by comparing the collected sensor data withdata obtained from the digital twin simulation system 13310. Forexample, consistent deviations or fluctuations between sensor dataobtained from the real-world element and the simulated element mayindicate malfunction of the respective sensor or another faultcondition.

In embodiments, the digital twin module 13420 may optimize features ofsmart container fleets, smart containers, and other shipping entitiesand/or environments through use of one or more simulated elements. Forexample, the digital twin module 13420 may evaluate effects of thesimulated elements within a digital twin of a smart container to quicklyand efficiently determine costs and/or benefits flowing from inclusion,exclusion, or substitution of real-world elements within the smartcontainer. The costs and benefits may include, for example,manufacturing costs, maintenance costs, efficiency (e.g., processoptimization to reduce waste or increase throughput), climateconsiderations (e.g., carbon footprint), lifespans, minimization ofcomponent faults, component downtime, or the like.

In embodiments, the digital twin I/O system 13308 may include one ormore software modules that are executed by one or more controllers ofone or more devices (e.g., server devices, user devices, and/ordistributed devices) to affect the described functions. The digital twinI/O system 13308 may include, for example, an input module, an outputmodule, and an adapter module.

In embodiments, the input module may obtain or import data from datasources in communication with the digital twin I/O system 13308, such asthe sensor system and the digital twin simulation system 13310. The datamay be immediately used by or stored within the digital twin module13420. The imported data may be ingested from data streams, databatches, in response to a triggering event, combinations thereof, andthe like. The input module may receive data in a format suitable totransfer, read, and/or write information within the digital twin module13420.

In embodiments, the output module may output or export data to othersystem components (e.g., the digital twin datastore 13316, the digitaltwin simulation system 13310, the intelligence service 13004, etc.),devices, and/or the client application 13324. The data may be output indata streams, data batches, in response to a triggering event (e.g., arequest), combinations thereof, and the like. The output module mayoutput data in a format that is suitable to be used or stored by thetarget element (e.g., one protocol for output to the client applicationand another protocol for the digital twin datastore 13316).

In embodiments, the adapter module may process and/or convert databetween the input module and the output module. In embodiments, theadapter module may convert and/or route data automatically (e.g., basedon data type) or in response to a received request (e.g., in response toinformation within the data).

In embodiments, the digital twin module 13420 may represent a set ofshipping workpiece elements in a digital twin, and the digital twinsimulation system 13310 simulates a set of physical interactions of aworker or shipping robot with the workpiece elements.

In embodiments, the digital twin simulation system 13310 may determineprocess outcomes for the simulated physical interactions accounting forsimulated human factors. For example, variations in workpiece throughputmay be modeled by the digital twin module 13420 including, for example,worker response times to events, worker fatigue, discontinuity withinworker actions (e.g., natural variations in human-movement speed,differing positioning times, etc.), effects of discontinuities ondownstream processes, and the like. In embodiments, individualizedworker interactions may be modeled using historical data that iscollected, acquired, and/or stored by the digital twin module 13420. Thesimulation may begin based on estimated amounts (e.g., worker age,industry averages, workplace expectations, etc.). The simulation mayalso individualize data for each worker (e.g., comparing estimatedamounts to collected worker-specific outcomes).

In embodiments, information relating to workers (e.g., fatigue rates,efficiency rates, and the like) may be determined by analyzingperformance of specific workers over time and modeling said performance.

In embodiments, the digital twin module 13420 includes plurality ofproximity sensors within the sensor array. The proximity sensors are ormay be configured to detect elements of a shipping entity or environmentthat are within a predetermined area. For example, proximity sensors mayinclude electromagnetic sensors, light sensors, and/or acoustic sensors.

The electromagnetic sensors are or may be configured to sense objects orinteractions via one or more electromagnetic fields (e.g., emittedelectromagnetic radiation or received electromagnetic radiation). Inembodiments, the electromagnetic sensors include inductive sensors(e.g., radio-frequency identification sensors), capacitive sensors(e.g., contact and contactless capacitive sensors), combinationsthereof, and the like.

The light sensors are or may be configured to sense objects orinteractions via electromagnetic radiation in, for example, thefar-infrared, near-infrared, optical, and/or ultraviolet spectra. Inembodiments, the light sensors may include image sensors (e.g.,charge-coupled devices and CMOS active-pixel sensors), photoelectricsensors (e.g., through-beam sensors, retroreflective sensors, anddiffuse sensors), combinations thereof, and the like. In embodiments,the light sensors may include liquid lens vision systems. Further, thelight sensors may be implemented as part of a system or subsystem, suchas a light detection and ranging (“LIDAR”) sensor.

The acoustic sensors are or may be configured to sense objects orinteractions via sound waves that are emitted and/or received by theacoustic sensors. In embodiments, the acoustic sensors may includeinfrasonic, sonic, and/or ultrasonic sensors. Further, the acousticsensors may be grouped as part of a system or subsystem, such as a soundnavigation and ranging (“SONAR”) sensor.

In embodiments, the digital twin module 13420 stores and collects datafrom a set of proximity sensors. The collected data may be stored, forexample, in the digital twin datastore 13316 for use by components thedigital twin module 13420 and/or visualization by a user. Such useand/or visualization may occur contemporaneously with or aftercollection of the data (e.g., during later analysis and/or optimizationof processes).

In embodiments, data collection may occur in response to a triggeringcondition. These triggering conditions may include, for example,expiration of a static or a dynamic predetermined interval, obtaining avalue short of or in excess of a static or dynamic value, receiving anautomatically generated request or instruction from the digital twinmodule 13420 or components thereof, interaction of an element with therespective sensor or sensors (e.g., in response to an object comingwithin a predetermined distance from the proximity sensor), interactionof a user with a digital twin (e.g., selection of a smart containerdigital twin, a sensor array digital twin, or a sensor digital twin),combinations thereof, and the like.

In some embodiments, the digital twin module 13420 collects and/orstores RFID data in response to interaction of a worker or robot with areal-world element. For example, in response to a robot interaction witha smart container cargo, the digital twin will collect and/or store RFIDdata from RFID sensors associated with the corresponding cargo.Additionally, or alternatively, robot interaction with a sensor-arraydigital twin will collect and/or store RFID data from RFID sensorswithin or associated with the corresponding sensor array. Similarly,robot interaction with a sensor digital twin will collect and/or storeRFID data from the corresponding sensor. The RFID data may includesuitable data attainable by RFID sensors such as proximate RFID tags,RFID tag position, authorized RFID tags, unauthorized RFID tags,unrecognized RFID tags, RFID type (e.g., active or passive), errorcodes, combinations thereof, and the like.

In embodiments, the digital twin module 13420 may further embed outputsfrom one or more devices within a corresponding digital twin. Inembodiments, the digital twin module 13420 embeds output from a set ofindividual-associated devices into a shipping digital twin. For example,the digital twin I/O system 13308 may receive information output fromone or more wearable devices or mobile devices (not shown) associatedwith an individual. The wearable devices may include image capturedevices (e.g., body cameras or augmented-reality headwear), navigationdevices (e.g., GPS devices, inertial guidance systems), motion trackers,acoustic capture devices (e.g., microphones), radiation detectors,combinations thereof, and the like.

In embodiments, upon receiving the output information, the digital twinI/O system 13308 routes the information to the digital twinconfiguration module 13318 to check and/or update the shipping digitaltwin and/or associated digital twins. Further, the digital twin module13420 may use the embedded output to determine characteristics of theshipping entity or environment.

In embodiments, the digital twin module 13420 embeds output from a LIDARpoint cloud system into a shipping digital twin. For example, thedigital twin I/O system 13308 may receive information output from one ormore LIDAR devices. The LIDAR devices are configured to provide aplurality of points having associated position data (e.g., coordinatesin absolute or relative x, y, and z values). Each of the plurality ofpoints may include further LIDAR attributes, such as intensity, returnnumber, total returns, laser color data, return color data, scan angle,scan direction, etc. The LIDAR devices may provide a point cloud thatincludes the plurality of points to the digital twin module 13420 via,for example, the digital twin I/O system 13308. Additionally, oralternatively, the digital twin module 13420 may receive a stream ofpoints and assemble the stream into a point cloud or may receive a pointcloud and assemble the received point cloud with existing point clouddata, map data, or three dimensional (3D)-model data.

In embodiments, upon receiving the output information, the digital twinI/O system 13308 routes the point cloud information to the digital twinconfiguration module 13318 to check and/or update the shipping digitaltwin and/or associated digital twins. In some embodiments, the digitaltwin module 13420 is further configured to determine closed-shapeobjects within the received LIDAR data. For example, the digital twinmodule 13420 may group a plurality of points within the point cloud asan object and, if necessary, estimate obstructed faces of objects (e.g.,a face of the object contacting or adjacent a floor or a face of theobject contacting or adjacent another object such as another piece ofequipment). The system may use such closed-shape objects to narrowsearch space for digital twins and thereby increase efficiency ofmatching algorithms (e.g., a shape-matching algorithm).

In embodiments, the digital twin module 13420 embeds output from asimultaneous location and mapping (“SLAM”) system in an environmentdigital twin. For example, the digital twin I/O system 13308 may receiveinformation output from the SLAM system, such as SLAM sensor, and embedthe received information within an environment digital twincorresponding to the location determined by the SLAM system. Inembodiments, upon receiving the output information from the SLAM system,the digital twin I/O system 13308 routes the information to the digitaltwin configuration module 13318 to check and/or update the shippingdigital twin and/or associated digital twins. Such updating providesdigital twins of non-connected elements automatically and without needof user interaction with the digital twin module 13420.

In embodiments, the digital twin module 13420 can leverage known digitaltwins to reduce computational requirements for the SLAM sensor by usingsuboptimal map-building algorithms. For example, the suboptimalmap-building algorithms may allow for a higher uncertainty toleranceusing simple bounded-region representations and identifying possibledigital twins. Additionally, or alternatively, the digital twin module13420 may use a bounded-region representation to limit the number ofdigital twins, analyze the group of potential twins for distinguishingfeatures, then perform higher precision analysis for the distinguishingfeatures to identify and/or eliminate categories of, groups of, orindividual digital twins and, in the event that no matching digital twinis found, perform a precision scan of only the remaining areas to bescanned.

In embodiments, the digital twin module 13420 may further reduce computerequired to build a location map by leveraging data captured from othersensors within the environment (e.g., captured images or video, radioimages, etc.) to perform an initial map-building process (e.g., a simplebounded-region map or other suitable photogrammetry method), associatedigital twins of known environmental objects with features of the simplebounded-region map to refine the simple bounded-region map, and performmore precise scans of the remaining simple bounded regions to furtherrefine the map. In some embodiments, the digital twin module 13420 maydetect objects within received mapping information and, for eachdetected object, determine whether the detected object corresponds to anexisting digital twin of a real-world-element. In response todetermining that the detected object does not correspond to an existingreal-world-element digital twin, the digital twin module 13420 may use,for example, the digital twin configuration module 13318 to generate anew digital twin corresponding to the detected object (e.g., adetected-object digital twin) and add the detected-object digital twinto the real-world-element digital twins within the digital twindatastore. Additionally, or alternatively, in response to determiningthat the detected object corresponds to an existing real-world-elementdigital twin, the digital twin module 13420 may update thereal-world-element digital twin to include new information detected bythe simultaneous location and mapping sensor, if any.

In embodiments, the digital twin module 13420 represents locations ofautonomously or remotely moveable elements and attributes thereof withina shipping digital twin. Such movable elements may include, for example,cargo, vehicles, autonomous vehicles, robots, etc. The locations of themoveable elements may be updated in response to a triggering condition.Such triggering conditions may include, for example, expiration of astatic or a dynamic predetermined interval, receiving an automaticallygenerated request or instruction from the digital twin module 13420 orcomponents thereof, interaction of an element with a respective sensoror sensors (e.g., in response to a worker or machine breaking a beam orcoming within a predetermined distance from a proximity sensor),interaction of a user with a digital twin (e.g., selection of a shippingdigital twin, a sensor array digital twin, or a sensor digital twin),combinations thereof, and the like.

In embodiments, the time intervals may be based on probability of therespective movable element having moved within a time period. Forexample, the time interval for updating a robot location may berelatively shorter for robots expected to move frequently (e.g., a robottasked with lifting and carrying cargo within and through a containerterminal) and relatively longer for robots expected to move infrequently(e.g., a robot tasked with monitoring a process). Additionally oralternatively, the time interval may be dynamically adjusted based onapplicable conditions, such as increasing the time interval when nomovable elements are detected, decreasing the time interval as or whenthe number of moveable elements within an environment increases (e.g.,increasing number of robots and robot interactions), increasing the timeinterval during periods of reduced activity, decreasing the timeinterval during periods of abnormal activity (e.g., inspections ormaintenance), decreasing the time interval when unexpected oruncharacteristic movement is detected (e.g., frequent movement by atypically sedentary element or coordinated movement, for example, ofrobots approaching an exit or moving cooperatively to carry a largeobject), combinations thereof, and the like. Further, the time intervalmay also include additional, semi-random acquisitions. For example,occasional mid-interval locations may be acquired by the digital twinmodule 13420 to reinforce or evaluate the efficacy of the particulartime interval.

In embodiments, the digital twin module 13420 may analyze data receivedfrom the digital twin I/O system 13308 to refine, remove, or addconditions. For example, the digital twin module 13420 may optimize datacollection times for movable elements that are updated more frequentlythan needed (e.g., multiple consecutive received positions beingidentical or within a predetermined margin of error).

In embodiments, the digital twin module 13420 may receive, identify,and/or store a set of states related to shipping entities orenvironments. The set of states may be, for example, data structuresthat include a plurality of attributes and a set of identifying criteriato uniquely identify each respective state. In embodiments, the set ofstates may correspond to states where it is desirable for the digitaltwin module 13420 to set or alter conditions of real-world elementsand/or the environment (e.g., increase/decrease monitoring intervals,alter operating conditions, etc.).

In embodiments, the set of states may further include, for example,minimum monitored attributes for each state, the set of identifyingcriteria for each state, and/or actions available to be taken orrecommended to be taken in response to each state. Such information maybe stored by, for example, the digital twin datastore 13316 or anotherdatastore. The set of states or portions thereof may be provided to,determined by, or altered by the digital twin module 13420. Further, theset of states may include data from disparate sources. For example,details to identify and/or respond to occurrence of a first state may beprovided to the digital twin module 13420 via user input, details toidentify and/or respond to occurrence of a second state may be providedto the digital twin module 13420 via an external system, details toidentify and/or respond to occurrence of a third state may be determinedby the digital twin module 13420 (e.g., via simulations or analysis ofprocess data), and details to identify and/or respond to occurrence of afourth state may be stored by the digital twin module 13420 and alteredas desired (e.g., in response to simulated occurrence of the state oranalysis of data collected during an occurrence of and response to thestate).

In embodiments, the plurality of attributes includes at least theattributes needed to identify the respective state. The plurality ofattributes may further include additional attributes that are or may bemonitored in determining the respective state, but are not needed toidentify the respective state. For example, the plurality of attributesfor a first state may include relevant information such as rotationalspeed, battery level, energy input, linear speed, acceleration,temperature, strain, torque, volume, weight, etc.

The set of identifying criteria may include information for each of theset of attributes to uniquely identify the respective state. Theidentifying criteria may include, for example, rules, thresholds,limits, ranges, logical values, conditions, comparisons, combinationsthereof, and the like.

The change in operating conditions or monitoring may be any suitablechange. For example, after identifying occurrence of a respective state,the digital twin module 13420 may increase or decrease monitoringintervals for a smart container (e.g., decreasing monitoring intervalsin response to a measured parameter differing from nominal operation)without altering operation of the smart container. Additionally, oralternatively, the digital twin module 13420 may alter operation of thesmart container (e.g., reduce speed or power input) without alteringmonitoring of the smart container. In further embodiments, the digitaltwin module 13420 may alter operation of the smart container (e.g.,reduce speed or power input) and alter monitoring intervals for thedevice (e.g., decreasing monitoring intervals).

In embodiments, a set of identified set of states related to shippingentities and environments that the digital twin module 13420 mayidentify and/or store for access by intelligent systems (e.g., theintelligence service 13004) or users of the digital twin module 13420,according to some embodiments of the present disclosure. The set ofstates may include operational states (e.g., suboptimal, normal,optimal, critical, or alarm operation of one or more components), excessor shortage states (e.g., supply-side or output-side quantities),combinations thereof, and the like.

In embodiments, the digital twin module 13420 may monitor attributes ofreal-world elements and/or digital twins to determine the respectivestate. The attributes may be, for example, operating conditions, setpoints, critical points, status indicators, other sensed information,combinations thereof, and the like. For example, the attributes mayinclude power input, operational speed, critical speed, and operationaltemperature of the monitored elements. While the illustrated exampleillustrates uniform monitored attributes, the monitored attributes maydiffer by target device (e.g., the digital twin module 13420 would notmonitor rotational speed for an object with no rotatable components).

Each of the set of states includes a set of identifying criteria meetingparticular criteria that are unique among the group of monitored set ofstates. The digital twin module 13420 may identify the overspeed state,for example, in response to the monitored attributes meeting a first setof identifying criteria (e.g., operational speed being higher than thecritical speed).

In response to determining that one or more set of states exists or hasoccurred, the digital twin module 13420 may update triggering conditionsfor one or more monitoring protocols, issue an alert or notification, ortrigger actions of subcomponents of the digital twin module 13420. Forexample, subcomponents of the digital twin module 13420 may take actionsto mitigate and/or evaluate impacts of the detected set of states. Whenattempting to take actions to mitigate impacts of the detected set ofstates on real-world elements, the digital twin module 13420 maydetermine whether instructions exist (e.g., are stored in the digitaltwin datastore 13316) or should be developed (e.g., developed viasimulation and intelligence services or via user or worker input).Further, the digital twin module 13420 may evaluate impacts of thedetected set of states, for example, concurrently with the mitigationactions or in response to determining that the digital twin module 13420has no stored mitigation instructions for the detected set of states.

In embodiments, the digital twin module 13420 employs the digital twinsimulation system 13310 to simulate one or more impacts, such asimmediate, upstream, downstream, and/or continuing effects, ofrecognized states. The digital twin simulation system 13310 may collectand/or be provided with values relevant to the evaluated set of states.In simulating the impact of the one or more set of states, the digitaltwin simulation system 13310 may recursively evaluate performancecharacteristics of affected digital twins until convergence is achieved.The digital twin simulation system 13310 may work, for example, intandem with the intelligence service 13004 to determine response actionsto alleviate, mitigate, inhibit, and/or prevent occurrence of the one ormore set of states. For example, the digital twin simulation system13310 may recursively simulate impacts of the one or more set of statesuntil achieving a desired fit (e.g., convergence is achieved), providethe simulated values to the intelligence service 13004 for evaluationand determination of potential actions, receive the potential actions,and/or evaluate impacts of each of the potential actions for arespective desired fit (e.g., cost functions for minimizing productiondisturbance, preserving critical components, minimizing maintenanceand/or downtime, optimizing system, worker, user, or personal safety,etc.).

In embodiments, the digital twin simulation system 13310 and theintelligence service 13004 may repeatedly share and update the simulatedvalues and response actions for each desired outcome until desiredconditions are met (e.g., convergence for each evaluated cost functionfor each evaluated action). The digital twin module 13420 may store theresults in the digital twin datastore 13316 for use in response todetermining that one or more set of states has occurred. Additionally,simulations and evaluations by the digital twin simulation system 13310and/or the intelligence service 13004 may occur in response tooccurrence or detection of the event.

In embodiments, simulations and evaluations are triggered only whenassociated actions are not present within the digital twin module 13420.In further embodiments, simulations and evaluations are performedconcurrently with use of stored actions to evaluate the efficacy oreffectiveness of the actions in real time and/or evaluate whetherfurther actions should be employed or whether unrecognized states mayhave occurred. In embodiments, the intelligence service 13004 may alsobe provided with notifications of instances of undesired actions with orwithout data on the undesired aspects or results of such actions tooptimize later evaluations.

In embodiments, the digital twin module 13420 evaluates and/orrepresents the impact of downtime of smart containers within a digitaltwin of a smart container fleet. For example, the digital twin module13420 may employ the digital twin simulation system 13310 to simulatethe immediate, upstream, downstream, and/or continuing effects of asmart container downtime state. The digital twin simulation system 13310may collect or be provided with performance-related values such asoptimal, suboptimal, and minimum performance requirements for elements(e.g., real-world elements and/or nested digital twins) within theaffected digital twins, and/or characteristics thereof that areavailable to the affected digital twins, effect on nested digital twins,redundant systems within the affected digital twins, combinationsthereof, and the like.

In embodiments, the digital twin module 13420 is configured to: simulateone or more operating parameters for the real-world elements in responseto the shipping entity or environment being supplied with givencharacteristics using the real-world-element digital twins; calculate amitigating action to be taken by one or more of the real-world elementsin response to being supplied with the contemporaneous characteristics;and actuate, in response to detecting the contemporaneouscharacteristics, the mitigating action. The calculation may be performedin response to detecting contemporaneous characteristics or operatingparameters falling outside of respective design parameters or may bedetermined via a simulation prior to detection of such characteristics.

Additionally, or alternatively, the digital twin module 13420 mayprovide alerts to one or more users or system elements in response todetecting states.

In embodiments, the digital twin I/O system 13308 includes a pathingmodule. The pathing module may ingest navigational data from theelements, provide and/or request navigational data to components of thedigital twin module 13420 (e.g., the digital twin simulation system13310, the digital twin dynamic model system 13312, and/or theintelligence service 13004), and/or output navigational data to elements(e.g., to the wearable devices). The navigational data may be collectedor estimated using, for example, historical data, guidance data providedto the elements, combinations thereof, and the like.

For example, the navigational data may be collected or estimated usinghistorical data stored by the digital twin module 13420. The historicaldata may include or be processed to provide information such asacquisition time, associated elements, polling intervals, taskperformed, laden or unladen conditions, whether prior guidance data wasprovided and/or followed, conditions of a shipping entity orenvironment, other elements within the shipping entity or environment,combinations thereof, and the like. The estimated data may be determinedusing one or more suitable pathing algorithms. For example, theestimated data may be calculated using suitable order-pickingalgorithms, suitable path-search algorithms, combinations thereof, andthe like. The order-picking algorithm may be, for example, a largest gapalgorithm, an s-shape algorithm, an aisle-by-aisle algorithm, a combinedalgorithm, combinations thereof, and the like. The path-searchalgorithms may be, for example, Dijkstra's algorithm, the A* algorithm,hierarchical path-finding algorithms, incremental path-findingalgorithms, any angle path-finding algorithms, flow field algorithms,combinations thereof, and the like.

In embodiments, the digital twin module 13420 ingests navigational datafor a set of smart containers for representation in a digital twin.Additionally, or alternatively, the digital twin module 13420 ingestsnavigational data for a set of mobile equipment assets of a shippingenvironment into a digital twin.

In embodiments, the digital twin module 13420 ingests a system formodeling traffic of mobile elements (e.g., smart containers, containerships, robots, trucks, trains, cargo, or the like) in a shipping digitaltwin. For example, the digital twin module 13420 may model trafficpatterns for a set of smart containers, mobile equipment assets, cargocombinations thereof, and the like. The traffic patterns may beestimated based on modeling traffic patterns from and historical dataand contemporaneous ingested data. Further, the traffic patterns may becontinuously or intermittently updated depending on conditions.

The digital twin module 13420 may alter traffic patterns (e.g., byproviding updated navigational data to one or more of the mobileelements) to achieve one or more predetermined criteria. Thepredetermined criteria may include, for example, increasing processefficiency, decreasing interactions between smart containers and mobileequipment assets, minimizing smart container path length, routing smartcontainers around paths or potential paths of persons, combinationsthereof, and the like.

In embodiments, the digital twin module 13420 may provide traffic dataand/or navigational information to mobile elements in a shipping digitaltwin. The navigational information may be provided as instructions orrule sets, displayed path data, or selective actuation of devices. Forexample, the digital twin module 13420 may provide a set of instructionsto a smart container to direct smart container to and/or along a desiredroute from an origin location to one or more specified locations alongthe route. The smart container may communicate updates to the systemincluding obstructions, reroutes, unexpected interactions with otherassets along the route.

In some embodiments, an ant-based system enables shipping entities,including smart containers, to lay a trail with one or more messages forother shipping containers and/or shipping entities, includingthemselves, to follow in later journeys. In embodiments, the messagesinclude information related to measurement collection.

In embodiments, the digital twin module 13420 includes designspecification information for representing a real-world element using adigital twin. The digital twin may correspond to an existing real-worldelement or a potential real-world element. The design specificationinformation may be received from one or more sources. For example, thedesign specification information may include design parameters set byuser input, determined by the digital twin module 13420 (e.g., the viadigital twin simulation system 13310), optimized by users or the digitaltwin simulation system 13310, combinations thereof, and the like. Thedigital twin simulation system 13310 may represent the designspecification information for the component to users, for example, via amonitor or a virtual reality headset. The design specificationinformation may be displayed schematically (e.g., as part of a processdiagram or table of information) or as part of an augmented reality orvirtual reality display. The design specification information may bedisplayed, for example, in response to a user interaction with thedigital twin module 13420 (e.g., via user selection of the element oruser selection to generally include design specification informationwithin displays). Additionally, or alternatively, the designspecification information may be displayed automatically, for example,upon the element coming within view of an augmented reality or virtualreality device. In embodiments, the displayed design specificationinformation may further include indicia of information source (e.g.,different displayed colors indicate user input versus digital twinmodule 13420 determination), indicia of mismatches (e.g., between designspecification information and operational information), combinationsthereof, and the like. In some embodiments, the digital twin module13420 may provide an augmented reality view that displays mismatchesbetween design parameters or expected parameters of real-world elementsto the wearer. The displayed information may correspond to real-worldelements that are not within the view of the wearer (e.g., elementswithin another room or obscured by machinery). This allows the worker toquickly and accurately troubleshoot mismatches to determine one or moresources for the mismatch. The cause of the mismatch may then bedetermined, for example, by the digital twin module 13420 and correctiveactions ordered. In example embodiments, a wearer may be able to viewmalfunctioning subcomponents of machines without removing occludingelements (e.g., housings or shields). Additionally, or alternatively,the wearer may be provided with instructions to repair the device, forexample, including display of the removal process (e.g., location offasteners to be removed), assemblies or subassemblies that should betransported to other areas for repair (e.g., dust-sensitive components),assemblies or subassemblies that need lubrication, and locations ofobjects for reassembly (e.g., storing location that the wearer hasplaced removed objects and directing the wearer or another wearer to thestored locations to expedite reassembly and minimize further disassemblyor missing parts in the reassembled element). This can expedite repairwork, minimize process impact, allow workers to disassemble andreassemble equipment (e.g., by coordinating disassembly without directcommunication between the workers), increase equipment longevity andreliability (e.g., by assuring that all components are properly replacedprior to placing back in service), combinations thereof, and the like.

In embodiments, the digital twin module 13420 may include, integrate,integrate with, manage, handle, link to, take input from, provide outputto, control, coordinate with, or otherwise interact with a digital twindynamic model system 13312. The digital twin dynamic model system 13312can update the properties of a set of digital twins of a set of shippingentities and/or environments, including properties of physical shippingassets, workers, processes, shipping facilities, warehouses, and thelike (or any of the other types of entities or environments described inthis disclosure or in the documents incorporated by reference herein) insuch a manner that the digital twins may represent those shippingentities and environments, and properties or attributes thereof, inreal-time or very near real-time. In some embodiments, the digital twindynamic model system 13312 may obtain sensor data received from a sensorsystem and may determine one or more properties of a shippingenvironment or a shipping entity based on the sensor data and based onone or more dynamic models.

In embodiments, the digital twin dynamic model system 13312 mayupdate/assign values of various properties in a digital twin and/or oneor more embedded digital twins, including, but not limited to, vibrationvalues, probability of failure values, probability of downtime values,cost of downtime values, pricing values, energy values, performancevalues, financial values, temperature values, humidity values, heat flowvalues, cargo load values, fluid flow values, radiation values,substance concentration values, velocity values, acceleration values,location values, pressure values, stress values, strain values, lightintensity values, sound level values, volume values, shapecharacteristics, material characteristics, and dimensions.

In embodiments, a digital twin may be comprised of (e.g., via reference)other embedded digital twins. For example, a digital twin of a containerterminal may include an embedded digital twin of a container ship andone or more embedded digital twins of one or more respective smartcontainers enclosed within the container ship. A digital twin may beembedded, for example, in the memory of a smart container that has anonboard IT system (e.g., the memory of an Onboard Diagnostic System,control system (e.g., SCADA system) or the like). Other non-limitingexamples of where a digital twin may be embedded include the following:on a wearable device of a worker; in memory on a local network asset,such as a switch, router, access point, or the like; in a cloudcomputing resource that is provisioned for an environment or entity; andon an asset tag or other memory structure that is dedicated to anentity.

In embodiments, the digital twin dynamic model system 13312 can updatethe properties of a digital twin and/or one or more embedded digitaltwins on behalf of a client application 13324 (such as the smartcontainer system 13000). In embodiments, a client application 13324 maybe the smart container system 13000, an application relating to ashipping component or environment (e.g., monitoring a shipping facilityor a component within, simulating a shipping environment, or the like).In embodiments, the client application 13324 may be used in connectionwith both fixed and mobile data collection systems. In embodiments, theclient application 13324 may be used in connection with IndustrialInternet of Things sensor system.

In embodiments, the digital twin dynamic model system 13312 leveragesdigital twin dynamic models 13374 to model the behavior of a shippingentity and/or environment. Dynamic models 13374 may enable digital twinsto represent physical reality, including the interactions of shippingentities, by using a limited number of measurements to enrich thedigital representation of a shipping entity and/or environment, such asbased on scientific principles. In embodiments, the dynamic models 13374are formulaic or mathematical models. In embodiments, the dynamic models13374 adhere to scientific laws, laws of nature, and formulas (e.g.,Newton's laws of motion, second law of thermodynamics, Bernoulli'sprinciple, ideal gas law, Dalton's law of partial pressures, Hooke's lawof elasticity, Fourier's law of heat conduction, Archimedes' principleof buoyancy, and the like). In embodiments, the dynamic models aremachine-learned models.

In embodiments, the digital twin module 13420 may have a digital twindynamic model datastore 13376 for storing dynamic models 13374 that maybe represented in digital twins. In embodiments, digital twin dynamicmodel datastore 13376 can be searchable and/or discoverable. Inembodiments, digital twin dynamic model datastore 13376 can containmetadata that allows a user to understand what characteristics a givendynamic model can handle, what inputs are required, what outputs areprovided, and the like. In some embodiments, digital twin dynamic modeldatastore 13376 can be hierarchical, such as where a model can bedeepened or made simpler based on the extent of available data and/orinputs, the granularity of the inputs, and/or situational factors (suchas where something becomes of high interest and a higher fidelity modelis accessed for a period of time).

In embodiments, a digital twin or digital representation of a shippingentity or environment may include a set of data structures thatcollectively define a set of properties of a represented physicalshipping asset, device, worker, process, facility, and/or environment,and/or possible behaviors thereof. In embodiments, the digital twindynamic model system 13312 may leverage the dynamic models 13374 toinform the set of data structures that collectively define a digitaltwin with real-time data values. The digital twin dynamic models 13374may receive one or more sensor measurements, Industrial Internet ofThings device data, and/or other suitable data as inputs and calculateone or more outputs based on the received data and one or more dynamicmodels 13374. The digital twin dynamic model system 13312 then uses theone or more outputs to update the digital twin data structures.

In one example, the set of properties of a digital twin of an shippingentity that may be updated by the digital twin dynamic model system13312 using dynamic models 13374 may include the vibrationcharacteristics of the shipping entity, temperature(s) of the shippingentity, the state of the shipping entity (e.g., a solid, liquid, orgas), the location of the shipping entity, the displacement of theshipping entity, the velocity of the shipping entity, the accelerationof the shipping entity, probability of downtime values associated withthe shipping entity, cost of downtime values associated with theshipping entity, financial information associated with the shippingentity, heat flow characteristics associated with the shipping entity,fluid flow rates associated with the shipping entity (e.g., fluid flowrates of a fluid flowing through a pipe), identifiers of other digitaltwins embedded within the digital twin of the shipping entity and/oridentifiers of digital twins embedding the digital twin of the shippingentity, and/or other suitable properties. Dynamic models 13374associated with a digital twin of an asset can be configured tocalculate, interpolate, extrapolate, and/or output values for such assetdigital twin properties based on input data collected from sensorsand/or devices disposed in the industrial setting and/or other suitabledata and subsequently populate the asset digital twin with thecalculated values.

In some embodiments, the set of properties of a digital twin of anshipping device that may be updated by the digital twin dynamic modelsystem 13312 using dynamic models 13374 may include the status of thedevice, a location of the device, the temperature(s) of a device, atrajectory of the device, identifiers of other digital twins that thedigital twin of the device is embedded within, embeds, is linked to,includes, integrates with, takes input from, provides output to, and/orinteracts with, and the like. Dynamic models 13374 associated with adigital twin of a device can be configured to calculate or output valuesfor these device digital twin properties based on input data andsubsequently update the device digital twin with the calculated values.

Example properties of a digital twin of a shipping environment that maybe updated by the digital twin dynamic model system 13312 using dynamicmodels 13374 may include the dimensions of the shipping environment, thetemperature(s) of the shipping environment, the humidity value(s) of theshipping environment, the fluid flow characteristics in the shippingenvironment, the heat flow characteristics of the shipping environment,the lighting characteristics of the shipping environment, the acousticcharacteristics of the shipping environment the physical objects in theenvironment, processes occurring in the shipping environment, currentsof the shipping environment (if a body of water), and the like. Dynamicmodels associated with a digital twin of a shipping environment can beconfigured to calculate or output these properties based on input datacollected from sensors and/or devices disposed in the shippingenvironment and/or other suitable data and subsequently populate theshipping environment digital twin with the calculated values.

In embodiments, dynamic models 13374 may adhere to physical limitationsthat define boundary conditions, constants or variables for digital twinmodeling. For example, the physical characterization of a digital twinof a shipping entity or shipping environment may include a gravityconstant (e.g., 9.8 m/s²), friction coefficients of surfaces, thermalcoefficients of materials, maximum temperatures of assets, maximum flowcapacities, and the like. Additionally, or alternatively, the dynamicmodels may adhere to laws of nature. For example, dynamic models mayadhere to the laws of thermodynamics, laws of motion, laws of fluiddynamics, laws of buoyancy, laws of heat transfer, laws of radiation,laws of quantum dynamics, and the like. In some embodiments, dynamicmodels may adhere to biological aging theories or mechanical agingprinciples. Thus, when the digital twin dynamic model system 13312facilitates a real-time digital representation, the digitalrepresentation may conform to dynamic models, such that the digitalrepresentations mimic real world conditions. In some embodiments, theoutput(s) from a dynamic model can be presented to a human user and/orcompared against real-world data to ensure convergence of the dynamicmodels with the real world. Furthermore, as dynamic models are basedpartly on assumptions, the properties of a digital twin may be improvedand/or corrected when a real-world behavior differs from that of thedigital twin. In embodiments, additional data collection and/orinstrumentation can be recommended based on the recognition that aninput is missing from a desired dynamic model, that a model in operationisn't working as expected (perhaps due to missing and/or faulty sensorinformation), that a different result is needed (such as due tosituational factors that make something of high interest), and the like.

Dynamic models may be obtained from a number of different sources. Insome embodiments, a user can upload a model created by the user or athird party. Additionally, or alternatively, the models may be createdon the digital twin system using a graphical user interface. The dynamicmodels may include bespoke models that are configured for a particularenvironment and/or set of shipping entities and/or agnostic models thatare applicable to similar types of digital twins. The dynamic models maybe machine-learned models. In embodiments, the dynamic models may bemachine-learned models provided by the intelligence service 13004.

In embodiments, digital twin dynamic model system 13312 leverages one ormore dynamic models 13374 to update a set of properties of a digitaltwin and/or one or more embedded digital twins on behalf of clientapplication 13324 based on the impact of collected sensor data fromsensor system, data collected from Internet of Things connected devices13338, and/or other suitable data in the set of dynamic models 13374that are used to enable the shipping digital twins. In embodiments, thedigital twin dynamic model system 13312 may be instructed to runspecific dynamic models using one or more digital twins that representphysical shipping entities, devices, workers, processes, and/or shippingenvironments that are managed, maintained, and/or monitored by theclient applications 13324.

In embodiments, the digital twin dynamic model system 13312 may obtaindata from other types of external data sources that are not necessarilyshipping data sources, but may provide data that can be used as inputdata for the dynamic models. For example, traffic data, trucking fleetdata, aviation data, road data, freight data, maritime data, weatherdata, news events, social media data, and the like may be collected,crawled, subscribed to, and the like to supplement sensor data,Industrial Internet of Things device data, and/or other data that isused by the dynamic models. In embodiments, the digital twin dynamicmodel system 13312 may obtain data from a machine vision module 13422.Machine vision module 13422 may use video and/or still images to providemeasurements (e.g., locations, statuses, and the like) that may be usedas inputs by the dynamic models.

In embodiments, the digital twin dynamic model system 13312 may feedthis data into one or more of the dynamic models discussed above toobtain one or more outputs. These outputs may include calculatedvibration characteristics, probability of failure values, probability ofdowntime values, cost of downtime values, time to failure values,temperature values, pressure values, humidity values, precipitationvalues, visibility values, air quality values, strain values, stressvalues, displacement values, velocity values, acceleration values,location values, performance values, financial values, pricing values,electrodynamic values, thermodynamic values, fluid flow rate values, andthe like. The client application 13324 may then initiate a digital twinvisualization event using the results obtained by the digital twindynamic model system 13312. In embodiments, the visualization may be aheat map visualization.

As illustrated by FIG. 148 , the digital twin dynamic model system 13312may receive requests to update one or more properties of digital twinsof shipping entities and/or environments such that the digital twinsrepresent the shipping entities and/or environments in real-time. Atstep A100, the digital twin dynamic model system 13312 receives arequest to update one or more properties of one or more the digitaltwins of shipping entities and/or environments. For example, the digitaltwin dynamic model system 13312 may receive the request from a clientapplication 13324 or from another process executed by the digital twinmodule 13420 (e.g., a predictive maintenance process). The request mayindicate the one or more properties and the digital twin or digitaltwins implicated by the request. In step A102, the digital twin dynamicmodel system 13312 determines the one or more digital twins required tofulfill the request and retrieves the one or more required digitaltwins, including any embedded digital twins, from digital twin datastore13316. At step A104, digital twin dynamic model system 13312 determinesone or more dynamic models required to fulfill the request and retrievesthe one or more required dynamic models from digital twin dynamic modeldatastore 13376. At step A106, the digital twin dynamic model system13312 selects one or more sensors from sensor system, data collectedfrom Internet of Things connected devices 13338, and/or other datasources from digital twin I/O system 13308 based on available datasources and the one or more required inputs of the dynamic model(s). Inembodiments, the data sources may be defined in the inputs required bythe one or more dynamic models or may be selected using a lookup table.At step A108, the digital twin dynamic model system 13312 retrieves theselected data from digital twin I/O system 13308. At step A110, digitaltwin dynamic model system 13312 runs the dynamic model(s) using theretrieved input data (e.g., velocity sensor data, image data, and thelike) as inputs and determines one or more output values based on thedynamic model(s) and the input data. At step A112, the digital twindynamic model system 13312 updates the values of one or more propertiesof the one or more digital twins based on the one or more outputs of thedynamic model(s).

In example embodiments, client application 13324 may be configured toprovide a digital representation and/or visualization of the digitaltwin of a shipping entity. In embodiments, the client application 13324may include one or more software modules that are executed by one ormore server devices. These software modules may be configured toquantify properties of the digital twin, model properties of a digitaltwin, and/or to visualize digital twin behaviors. In embodiments, thesesoftware modules may enable a user to select a particular digital twinbehavior visualization for viewing. In embodiments, these softwaremodules may enable a user to select to view a digital twin behaviorvisualization playback. In some embodiments, the client application13324 may provide a selected behavior visualization to digital twindynamic model system 13312.

In embodiments, the digital twin dynamic model system 13312 may receiverequests from client application 13324 to update properties of a digitaltwin in order to enable a digital representation of a shipping entityand/or environment wherein the real-time digital representation is avisualization of the digital twin. In embodiments, a digital twin may berendered by a computing device, such that a human user can view thedigital representations of real-world shipping entities, devices,workers, processes and/or environments. For example, the digital twinmay be rendered an outcome to a display device. In embodiments, dynamicmodel outputs and/or related data may be overlaid on the rendering ofthe digital twin. In embodiments, dynamic model outputs and/or relatedinformation may appear with the rendering of the digital twin in adisplay interface. In embodiments, the related information may includereal-time video footage associated with the real-world entityrepresented by the digital twin. In embodiments, the related informationmay be graphical information. In embodiments, graphical information maydepict motion, wherein a user is enabled to select a view of thegraphical information in the x, y, and z dimensions. In embodiments, therelated information may be cost data, including cost of downtime per daydata, cost of repair data, cost of new part data, cost of new machinedata, and the like. In embodiments, related information may beprobability of downtime data, probability of failure data, and the like.In embodiments, related information may be time to failure data.

In embodiments, the related information may be recommendations and/orinsights. For example, recommendations or insights received from theintelligence service related to a smart container may appear with therendering of the digital twin of a smart container in a displayinterface.

In embodiments, clicking, touching, or otherwise interacting with thedigital twin rendered in the display interface can allow a user to“drill down” and see underlying subsystems or processes and/or embeddeddigital twins. In embodiments, clicking, touching, or otherwiseinteracting with information related to the digital twin rendered in thedisplay interface can allow a user to “drill down” and see underlyinginformation

In some embodiments, the digital twin dynamic model system 13312 mayreceive requests from client application 13324 to update properties of adigital twin in order to enable a digital representation of shippingentities and/or environments wherein the digital representation is aheat map visualization of the digital twin. In embodiments, a platformis provided having heat maps displaying collected data from the sensorsystem, Internet of Things connected devices 13338, and data outputsfrom dynamic models 13374 for providing input to a display interface. Inembodiments, the heat map interface is provided as an output for digitaltwin data, such as for handling and providing information forvisualization of various sensor data, dynamic model output data, andother data (such as map data, analog sensor data, and other data), suchas to another system, such as a mobile device, tablet, dashboard,computer, AR/VR device, or the like. A digital twin representation maybe provided in a form factor (e.g., user device, VR-enabled device,AR-enabled device, or the like) suitable for delivering visual input toa user, such as the presentation of a map that includes indicators oflevels of analog sensor data, digital sensor data, and output valuesfrom the dynamic models. In embodiments, signals from various sensors orinput sources (or selective combinations, permutations, mixes, and thelike) as well as data determined by the digital twin dynamic modelsystem 13312 may provide input data to a heat map. Coordinates mayinclude real world location coordinates (such as geo-location orlocation on a map), as well as other coordinates, such as time-basedcoordinates, frequency-based coordinates, or other coordinates thatallow for representation of analog sensor signals, digital signals,dynamic model outputs, input source information, and variouscombinations, and/or in a map-based visualization, such that colors mayrepresent varying levels of input along the relevant dimensions. Forexample, among many other possibilities, if a container terminal isoperating at a critical level state (e.g., due to heavy traffic ordelays), the heat map interface may alert a user by showing thecontainer port in orange. In the example of a heat map, clicking,touching, or otherwise interacting with the heat map can allow a user todrill down and see underlying container ships, dynamic model outputs, orother input data that is used as an input to the heat map display. Inother examples, such as ones where a digital twin is displayed in a VRor AR environment, if a smart container machine component is vibratingoutside of normal operation, a haptic interface may induce vibrationwhen a user touches a representation of the machine component, or if amachine component is operating in an unsafe manner, a directional soundsignal may direct a user's attention toward the machine in digital twin,such as by playing in a particular speaker of a headset or other soundsystem.

In embodiments, the digital twin dynamic model system 13312 may take aset of ambient environmental data and/or other data and automaticallyupdate a set of properties of a digital twin of a shipping entity orenvironment based on the impact of the environmental data and/or otherdata in the set of dynamic models 13374 that are used to enable thedigital twin. Ambient environmental data may include temperature data,pressure data, humidity data, wind data, rainfall data, tide data, stormsurge data, cloud cover data, current data, snowfall data, visibilitydata, water level data, and the like. Additionally, or alternatively,the digital twin dynamic model system 13312 may use a set ofenvironmental data measurements collected by a set of Internet of Thingsconnected devices 13338 disposed in an industrial setting as inputs forthe set of dynamic models 13374 that are used to enable the digitaltwin. For example, digital twin dynamic model system 13312 may feed thedynamic models 13374 data collected, handled or exchanged by Internet ofThings connected devices, such as cameras, monitors, embedded sensors,mobile devices, diagnostic devices and systems, instrumentation systems,telematics systems, and the like, such as for monitoring variousparameters and features of machines, devices, components, parts,operations, functions, conditions, states, events, workflows and otherelements (collectively encompassed by the term “states”) of shippingenvironments. Other examples of Internet of Things connected devicesinclude smart fire alarms, smart security systems, smart air qualitymonitors, smart/learning thermostats, and smart lighting systems.

FIG. 149 illustrates an example embodiment of a method for updating aset of cost of downtime values in the digital twin of a smart container.In the present example, the digital twin dynamic model system 13312 mayreceive requests from a client application 13324 to populate real-timecost of downtime values associated with a smart container in a smartcontainer fleet digital twin. At step B200, digital twin dynamic modelsystem 13312 receives a request from the client application 13324 toupdate one or more cost of downtime values of the smart containerdigital twin 13504 and any embedded digital twins (e.g., robots, cargo,and the like) from the client application 13324. Next, in step B202, thedigital twin dynamic model system 13312 determines the one or moredigital twins required to fulfill the request and retrieves the one ormore required digital twins. In this example, the digital twin dynamicmodel system 13312 may retrieve the digital twins of the fleet, thesmart containers, and any other embedded digital twins from digital twindatastore 13316. At step B204, digital twin dynamic model system 13312determines one or more dynamic models required to fulfill the requestand retrieves the one or more required dynamic models from dynamic modeldatastore 13316. At step B206, the digital twin dynamic model system13312 selects dynamic model input data sources (e.g., one or moresensors from sensor system and/or any other suitable data) based onavailable data sources (e.g., available sensors from a set of sensors insensor system) and the one or more required inputs of the dynamicmodel(s) via digital twin I/O system 13308. In the present example, theretrieved dynamic model(s) may be configured to take historical downtimedata and operational data as input and output data representing cost ofdowntime per day for smart containers in the smart container fleet. Atstep B208, digital twin dynamic model system 13312 retrieves historicaldowntime data and operational data from digital twin I/O system 13308.At step B210, digital twin dynamic model system 13312 runs the dynamicmodel(s) using the retrieved data as input and calculates one or moreoutputs that represent cost of downtime per day for smart containers inthe smart container fleet. Next, at step B212, the digital twin dynamicmodel system 13312 updates one or more cost of downtime values of thesmart container digital twin 13504 and embedded digital twins based onthe one or more outputs of the dynamic model(s).

In embodiments, the smart container system 13000 includes a roboticprocess automation (RPA) module 13416 configured to automate internalshipping workflows based on robotic process automation. The RPA module13416 may develop a programmatic interface to a user interface of anexternal system such as devices, programs, networks, databases, and thelike. The RPA module 13416 is configured to allow the smart containersystem 13000 to interface with an external system without using anapplication programming interface (API), or in addition to an API. TheRPA module 13416 may develop an action list by watching a user perform atask in a graphical user interface (GUI) and recording the tasks in theaction list. The RPA module 13416 may automate a workflow by repeatingtasks of the action list in the GUI.

In some embodiments, the RPA module 13416 may include and/or communicatewith an intelligence service 13004 configured to perform robotic processautomation processes. The intelligence service 13004 may employ one ormore machine learning techniques to develop one or more machine-learnedmodels. The machine-learned models may be capable of developing,defining, and/or implementing RPA-based programmatic interfaces tofacilitate interfacing of the system 13000 with one or more externaldevices.

The RPA module 13416 may be necessary for the smart container system13000 to communicate with an external system that does not have an APIor that has an outdated API. For example, the RPA module 13416 may allowthe smart container system 13000 to interface with an older externaldevice that does not include an API or that has an outdated API. The RPAmodule 13416 may allow the smart container system 13000 to interfacewith an external system similarly to how a user would interface with theexternal system, such as via a user interface of the external system. Insome embodiments, the RPA module 13416 allows the smart container system13000 to emulate an action and/or a series of actions performable by auser to interface with an external system. Examples of programmaticinterfacing by the RPA module 13416 to interface with an external systeminclude manipulation of markup language, emulating computer mousemovements and/or “clicking on” one or more elements of a user interface,entering information into fillable fields and submitting the informationvia a client program and/or portal, and transmitting digital signals toan external system that appear to be from sent from a user device.

In some embodiments, the RPA module 13416 may be configured tofacilitate communicating with new and/or updated external systems. Whena new external system is developed or an external system is updated, theRPA module 13416 may develop a new and/or updated programmatic interfaceto facilitate interfacing with the new and/or updated external system bythe smart container system 13000 in a manner that is consistent withinterfacing with an outdated external device, i.e., the external deviceprior to release of the new and/or updated external system. For example,the RPA module 13416 may be configured to provide inputs to the outdatedexternal device, provide inputs to the new and/or updated externaldevice, compare related outputs, and adjust inputs to the new and/orupdated external device such that the smart container system 13000 mayinterface with the new and/or updated external device in a mannerconsistent with how the smart container system 13000 interfaced with theoutdated external device.

In some embodiments, the RPA module 13416 may act as an API to outdatedand/or external systems. The RPA module 13416 may be configured suchthat the smart container system 13000 is externally represented ashaving an API capable of interfacing with one or more external devicesor otherwise being capable of programmatically handling signalstransmitted by external devices, wherein the RPA module 13416 hasdeveloped a programmatic interface for handling such requests other thanan API. For example, an outdated external system may be configured tocommunicate via a series of signals understood by an outdated API. TheRPA module 13416 may configure the smart container system 13000 to actas if the smart container system 13000 includes the outdated API.

In some embodiments, the RPA module 13416 may be configured to provide auser interface for use by one or more users of the smart containersystem 13000. The intelligence service 13004 may, by one or more machinelearning methods, create a user interface that allows a user tointerface with one or more components and/or functions of the smartcontainer system 13000. The RPA module 13416 may use robotic processautomation techniques to operate the user interface created by theintelligence service 13004. The intelligence service 13004 maydynamically create and/or adjust the user interface according tovariables such as changing demand conditions, new and/or modifiedfunctions of the smart container system 13000, new and/or modifiedconditions of systems external to the smart container system 13000, andthe like. Examples of new and/or modified conditions of systems externalto the smart container system 13000 may include changes to third-partyservice offerings, regulatory changes, and the like.

In some embodiments, the RPA module 13416 may be configured to avoiddetection of robotic process automation by systems external to the smartcontainer system 13000. Some of the external systems may be designed toattempt to detect, when the external system is communicating with asystem using robotic process automation, such as the smart containersystem 13000. Upon detecting that the smart container system 13000 isusing robotic process automation, the external system may restrict,eliminate, or modify communication capabilities of the smart containersystem 13000 with the external system. The RPA module 13416 may emulatehuman interfacing with the external system to “trick” the externalsystem into believing that the RPA module 13416 is a human user to avoiddetection of the robotic process automation and avoid restriction orelimination of communication by the external system. The RPA module13416 may avoid detection by, for example, dynamically changing paths ofinteraction with the external system, interacting with user interfaceelements with inconsistent timing, making human-like errors such as“misclicks” or “typos,” and the like.

In some embodiments, the intelligence service 13004 may be configured tocreate a machine-learned model for avoiding detection of robotic processautomation. The machine-learned model may be created by using data frominteraction with one or more graphical interfaces by real human beingsand developing robotic process automation techniques that emulate waysin which real humans interface with the one or more graphical userinterfaces. For example, training data may include mouse and/or touchtimings and accuracy, typing speed and accuracy, elements of thegraphical user interface used, and the like.

In some embodiments, the RPA module 13416 may be configured to validatedata transmitted to and/or received from external systems. The RPAmodule 13416 may validate one or more of data transmitted to the smartcontainer system 13000 by users of the external system, data transmittedto the smart container system 13000 by users of the smart containersystem 13000, and/or data transmitted to the external system by users ofthe smart container system 13000. The RPA module 13416 may validate databy one or more of performing optical character recognition, performingimage recognition and/or processing, identifying data stored onwebpages, receiving data from a backend database of the external system,receiving data from a backend database of the smart container system13000, and the like.

In some embodiments, the intelligence service 13004 may be configured todevelop one or more machine-learned models for data validation. Forexample, the intelligence service 13004 may use data transmitted byusers and/or data received from one or more databases and/or sourcesexternal to the smart container system 13000 as training data to “learn”to identify valid data. The intelligence service 13004 may transmit theone or more machine-learned models for data validation to the RPA module13416. The RPA module 13416 may implement the one or moremachine-learned models for data validation.

In some embodiments, the RPA module 13416 may be configured tofacilitate validation of processes performed by the RPA module 13416.The RPA module 13416 may create a plurality of process validation logsas the RPA module 13416 performs one or more processes related to thesmart container system 13000 and/or external systems on behalf of one ormore users. The process validation logs may include one or more oftimestamps, transaction receipts, user interface screenshots, or anyother suitable data entry, file, and the like for providing validationof processes performed by the RPA module 13416. The RPA module 13416 maystore the process validation logs in one or more databases and maytransmit the process validation logs to the smart container system 13000and/or users of the smart container system 13000. The RPA module 13416may transmit the process validation logs automatically according to aschedule, upon demand by a user of the smart container system 13000,upon one or more conditions being true, and the like.

In some embodiments, the RPA module 13416 may be configured to adjustbehavior of the robotic process automation in response to feedbackacquired via one or both of data validation and process validation. Auser of the smart container system 13000 may view validations of dataprovided by the RPA module 13416 and, in response to the validations ofdata, instruct the RPA module 13416 to adjust behavior of the RPA module13416. A user of the smart container system 13000 may view one or moreof the process validation logs and, in response to the one or moreprocess validation logs, instruct the RPA module 13416 to adjustbehavior of the RPA module 13416. Adjustment of behavior of the RPAmodule 13416 may include using different robotic process automationtechniques to perform features of the RPA module 13416, such as, forexample, changing RPA-based user interface elements presented to usersof the smart container system 13000, adjusting how the RPA module 13416interfaces with one or more external systems, and any other suitableadjustment by the RPA module 13416.

In some embodiments, the intelligence service 13004 may use datavalidation information and/or feedback, process validation logs, or acombination thereof as training data. The intelligence service 13004 maytrain one or more machine-learned models to influence, adjust, and/orotherwise control behavior of the RPA module 13416 based upon the datavalidation information and/or feedback, process validation logs, or acombination thereof.

In some embodiments, the RPA module 13416 may be configured to performimage processing to recognize images in graphical user interfaces withwhich the RPA module 13416 interfaces. Graphical user interfaces ofexternal systems with which the RPA module 13416 interfaces may bechanged and/or updated, thereby potentially disrupting robotic processautomation-based interfacing with the GUI. The RPA module 13416 mayautomatically detect changes to the GUI via image recognition and/orimage processing. The RPA module 13416 may automatically update roboticprocess automation-based interfacing with the updated GUI to facilitatecontinued interfacing with the updated GUI and avoid errors orinterruptions in communication with the external system.

In some embodiments, the intelligence service 13004 may use imageprocess optimization to use one or more machine-learned models toautomatically correct robotic process automation-based interfacing withthe external system of the RPA module 13416. For example, theintelligence service 13004 may use a plurality of GUIs having images astraining data to create a machine-learned model capable of automaticallydetecting changes in GUIs of external systems and determining how toadjust robotic process automation of the RPA module 13416 such that theRPA module 13416 may automatically continue interfacing with the GUI inlight of a change to the GUI.

In some embodiments, the RPA module 13416 may be configured to develop ahuman training system for instructing humans to interface with one ormore user interfaces of the smart container system 13000 and/or one ormore external systems. The human training system may teach one or morehuman users a plurality of actions and/or techniques employed by the RPAmodule 13416 to interface with the one or more user interfaces such thatthe human users may perform tasks similarly to the RPA module 13416. Thehuman training system may include one or more documents, videos,tutorials, and the like for facilitating human learning of actionsand/or techniques for interfacing with the user interfaces.

In some embodiments, the RPA module 13416 may be configured to processand document success criteria of robotic process automation implementedby the RPA module 13416. The processed and documented success criteriais descriptive such that a human user of the smart container system13000 and/or the RPA module 13416 may use the processed and documentedsuccess criteria to understand one or more process steps and/oralgorithms used by the RPA module 13416 to facilitate interfacing withexternal systems and/or to automate internal marketplace workflows ofthe smart container system 13000.

In some embodiments, the RPA module 13416 may implement gamification ofrobotic process automation capabilities of the smart container system13000. The gamification of robotic process automation capabilities mayinclude awarding points to users for performing tasks desirable tooperation of the smart container system 13000 and/or desirable forimprovement of robotic process automation operations of the smartcontainer system 13000. For example, points may be awarded foraugmentation of a robotic process automation algorithm. Users who havebeen awarded points may compete with one another, and digital and/orphysical prizes may be awarded to users who have achieved one or morepoint thresholds and/or have ranked above one or more other users on apoints leaderboard.

In embodiments, the smart container system 13000 includes an edge deviceconfigured to perform edge computation and intelligence. In someembodiments, edge computation and intelligence may include performingone or both of data processing and data storage in an area that isphysically close to where the processed and/or stored data is needed. Insome embodiments, the smart container system 13000 may include aplurality of edge devices. By way of example, the edge device may be arouter, a routing switch, an integrated access device, a multiplexer, alocal area network (LAN) and/or wide area network (WAN) access device,an Internet of Things device, a smart container, and/or any othersuitable device. In some embodiments, edge computation and intelligencemay include performing data processing and/or data filtering. Theprocessed and/or filtered data may be transmitted directly to devicesthat will use the processed and/or filtered data. The processed and/orfiltered data may be transmitted along transmission paths with lesscongestion than general-purpose or high-traffic data transmission paths.Transmission of the processed and/or filtered data may use lowerbandwidth than would transmission of unprocessed and/or unfiltered data.

In some embodiments, the edge device may implement local edgeintelligence to anticipate relevant shipping factors using data receivedby and/or stored by the edge device. The edge device may be directed tocollecting and processing data related to one or more of a particularsmart container, class of smart containers, shippers, class of shippers,shipping lines, class of shipping lines, container ports, class ofcontainer ports, and the like. In some embodiments, the edge device maybe situated physically near a remote container port or shipping hubarea. For example, the edge device may be positioned and configured tocollect data regarding performance related to a particular type of smartcontainer in a geographical region. The edge device may perform dataprocessing, analytics, filtering, trend finding, prediction making, andthe like related to the data and may send processing results, analytics,filtered data, trends, predictions, etc. or portions thereof to a morecentralized server, processor and/or data center within the smartcontainer system 13000.

In some embodiments, the edge device may be configured to performdecision making while being physically and/or electronically isolatedfrom some or all other components of the smart container system 13000.Herein, electronic isolation may mean or include being temporarilyunable to communicate with one or more other systems, devices,components, etc. The edge device may make decisions based upon outputsand/or conclusions drawn from the data processing, analytics, filtering,trend finding, prediction making, etc. related to data received by theedge device. Examples of decisions made by the edge device includewhether to validate one or more pieces of data, whether to validate auser of the smart container system 13000 or a portion thereof, whether afreight storage and/or transportation service order has been executed,and the like. The edge device may transmit data related to decisionsmade by the edge device to other components of the smart containersystem 13000.

In some embodiments, in cases where the edge device is temporarilyelectronically isolated from other components of the smart containersystem 13000, the edge device may make decisions on behalf of othercomponents of the smart container system 13000, and may have thedecisions audited, evaluated, and/or recorded by other components of thesmart container system 13000 upon being reconnected with the othercomponents of the smart container system 13000. The edge device may berestricted from making some decisions in absence of connection to and/oroversight by other components of the smart container system 13000.Examples of restricted decisions may include decisions related toshipping transactions where confidentiality and/or security are ofconcern, where sensitive cargo is to be shipped, and the like.

In some embodiments, the edge device may store a copy of a distributedledger, the distributed ledger containing information related to one ormore smart containers, smart container fleets, and/or shippingtransactions managed by the smart container system 13000. Thedistributed ledger may be a cryptographic ledger, such as a blockchain.The edge device may write blocks to the distributed ledger containingsmart container information and may have the blocks verified bycomparison with copies of the distributed ledger stored on othercomponents of the smart container system 13000.

In some embodiments, the smart container system 13000 may include aledger management system configured to manage a network of devices, suchas edge devices, that store copies of the distributed ledger. Thedevices that store copies of the distributed ledger may be configured totransmit copies stored thereon to the ledger management system foraggregation, comparison, and/or validation. The ledger management systemmay establish a whitelist of trusted parties and/or devices, a blacklistof untrusted parties and/or devices, or a combination thereof. Theledger management system may assign permissions to particular users,devices, and the like. Versions of the distributed ledger may becompared to prevent duplicate transactions such as the sale of multiplecopies of a unique good. In embodiments, where the smart containersystem 13000 includes a plurality of edge devices that may each store acopy of the distributed ledger and may compare copies against oneanother with respect to validation of blocks and addition of new blocksby and/or all of the edge devices.

In some embodiments, the smart container system 13000 may implement oneor more distributed update management algorithms for updatingdistributed devices such as the edge device. The distributed updatemanagement algorithm may include one or more procedures for how and whento roll out updates to the distributed devices. The smart containersystem 13000 may manage versions of edge computation software via thedistributed update management algorithms. The distributed devices mayreceive updates directly from the smart container system 13000, maytransmit updates to one another, or a combination thereof.

In some embodiments, wherein the smart container system 13000 includes aplurality of edge devices, the edge devices may communicate with oneanother to record and/or validate shipping data. The edge devices mayalso communicate data with one another related to one or more smartcontainers, smart container fleets, container ships, containerterminals, shipping yards, charging stations, regions, users, shippers,shipping lines, third parties, and the like. An edge device of theplurality of edge devices may communicate such information when able incases where an edge device is electronically isolated from other edgedevices and/or other components of the smart container system 13000.

In some embodiments, a first edge device that is electrically isolatedand is assigned to facilitate a smart container repair may be supportedby a second edge device. The second edge device may be assigned tofacilitate the same repair in case the first edge device fails tofacilitate the repair and/or is out of communication with othercomponents of the smart container system 13000 for an extended period oftime such that facilitation of repair by the first edge device isunverifiable. Upon reentering communication range, the first edge devicemay update the second edge device and/or other components of the smartcontainer system 13000 with maintenance operations that took place whilethe first edge device was electronically isolated.

In some embodiments, the smart container system 13000 may implement ahardware failure algorithm configured to make decisions when one or morecomponents of the smart container system 13000, such as the edge device,ceases operation and/or is otherwise unable to completely operateproperly. The hardware failure algorithm may include, for example,assigning an edge device to overtake operations that had been previouslyassigned to a now malfunctioning or nonfunctioning edge device.

In some embodiments, the smart container system 13000 may implement adata routing algorithm configured to optimize flow of data transmittedto and/or from the edge device, other components of the fleet system,external systems, or a combination thereof. The edge device may includeone or more signal amplifiers, signal repeaters, digital filters, analogfilters, digital-to-analog converters, analog-to-digital convertersand/or antennae configured to optimize the flow of data. In someembodiments, the network enhancement system may include a wirelessrepeater system such as is disclosed by U.S. Pat. No. 7,623,826 toPergal, the entirety of which is hereby incorporated by reference. Theedge device may optimize the flow of data by, for example, filteringdata, repeating data transmission, amplifying data transmission,adjusting one or more sampling rates and/or transmission rates, andimplementing one or more data communication protocols. In embodiments,the edge device may transmit a first portion of data over a first pathof the plurality of data paths and a second portion of data over asecond path of the plurality of data paths. The edge device maydetermine that one or more data paths, such as the first data path, thesecond data path, and/or other data paths, are advantageous fortransmission of one or more portions of data. The edge device may makedeterminations of advantageous data paths based upon one or morenetworking variables, such as one or more types of data beingtransmitted, one or more protocols being suitable for transmission,present and/or anticipated network congestion, timing of datatransmission, present and/or anticipated volumes of data being or to betransmitted, and the like. Protocols suitable for transmission mayinclude transmission control protocol (TCP), user datagram protocol(UDP), and the like. In some embodiments, the edge device may beconfigured to implement a method for data communication such as isdisclosed by U.S. Pat. No. 9,979,664 to Ho et al., the entirety of whichis hereby incorporated by reference.

In embodiments, the smart container system 13000 includes a digital twinmodule 13420 configured to receive data from the edge device and createa digital twin from the received data. The digital twin created by thedigital twin module 13420 may be a digital twin of one or more of asmart container fleet, a smart container, a fleet manager, a containerterminal, a shipping yard, a container ship, a shipper, cargo, and thelike, and may be created using any or all of the data received from theedge device. The edge device may transmit shipping-related data, such asdata related to a smart container, smart container cargo, a shipper, acontainer port, and the like, or a combination thereof. In embodiments,where the smart container system 13000 includes a plurality of edgedevices, the digital twin module 13420 may create the digital twin basedon data received from multiple of the plurality of edge devices.

In some embodiments, the edge device may be configured to facilitatepre-calculation and aggregation of data for a set of user-configuredreports. The user-configured reports may be integrated into the digitaltwin created by the digital twin module 13420. A user of the smartcontainer system 13000 may define one or more parameters of theuser-configured report to be included in the digital twin. The edgedevice may implement one or more data processing and/or filteringaccording to the parameters of the user-configured report. The edgedevice may transmit processed and/or filtered data relevant to theuser-configured report parameters to the digital twin module 13420. Uponreceiving the processed and/or filtered data, the digital twin module13420 may create the digital twin including the user-configured reportusing the received data and present the digital twin to the user.

In some embodiments, the edge device may be configured to collect andprocess data for use by one or more artificial intelligence (AI)systems. The AI systems may include the intelligence service 13004, oneor more artificial intelligence systems configured to facilitatecreation of the digital twin by the digital twin module 13420, and/orany other artificial intelligence systems connected to and/or includedin the smart container system 13000. The edge device may be configuredto collect and process and/or filter data such that the data is suitablefor use by the one or more AI systems. An example of processed and/orfiltered data collected by the edge device for use by the one or more AIsystems is training data for use in training one or more machine-learnedmodels.

In some embodiments, the edge device may be configured to locally storedata related to creation of the digital twin by the digital twin module13420. In cases where the digital twin is related to a particularregion, shipper, smart container, fleet, container port, ship, or thelike, the edge device may be particularly positioned to collect andstore data for use in populating the digital twin, for example, by beingpositioned nearby to the particular region, shipper, smart container,fleet, container port, ship, etc. The edge device may receive, process,filter, organize, and/or store data prior to transmission of the data tothe digital twin module 13420 such that the data is relevant to and/orsuitable for population of the digital twin. In some embodiments, theedge device may be configured to organize timing of transmission of dataused to populate the digital twin. The edge device may implement one ormore algorithms configured to measure and/or predict congestion of oneor more network paths and/or routes and may perform organization oftiming of transmission data based on the measurements and/or predictionsof the congestion. The edge device may in some cases prioritizetransmission of some types of data over others, such as according topriorities set by a user or by the digital twin module 13420. Forexample, the edge device may schedule regular transmissions oflow-priority information during evening hours, when congestion is low,and may transmit high-priority information substantially immediatelyupon receiving the high-priority information and/or receiving a requestfor the high-priority information. In some embodiments, the edge devicemay be configured to select a data protocol for transmission of dataused to populate the digital twin. The edge device may implement one ormore algorithms configured to select one or more optimal network pathsand/or routes and may select the data transmission protocol based on themeasurements and/or predictions of the congestion.

In some embodiments, the edge device may be in communication with andreceive data from a plurality of sensors. The edge device may beconfigured to intelligently multiplex alternative sensors amongavailable sensors in a shipping environment for the digital twin.

Digital Product Networks

FIG. 150 illustrates communications between entities in an exampledigital product network 14000 according to some embodiments of thepresent disclosure. In some embodiments, digital product networks arecommunicatively coupled entities of a value chain network that mayprovide data related to product level behavior, product level usage,environmental data, data that is processed at the product level, and thelike. For example, the product data may relate to at least one ofsensors, vibration, humidity, temperature, pressure, proximity, level,accelerometers, gyroscope, infrared sensors, MEMs, liquid lenses, shock,security, machine, product, pneumatic, conductive, state dependentfrequency monitor, ultrasonic, capacitance, or microwave.

An intelligence layer of the digital product network may then use theproduct level data to enable companies to solve challenges associatedwith customer demands for quality, efficiency, response, agility, andtransparency. For example, the product level behavior and usage data maybe combined with third-party sources to be analyzed and manipulated byartificial intelligence (AI) systems, machine learning (ML) systems,digital twin systems, robotic process automation (RPA) systems, etc.Data processing techniques may be related to at least one ofobfuscation, forecasting, simulation, transformation, automation,reporting, matching, stream processing, event processing and policy,dispatch and orchestration, analytics and algorithms, or machinelearning.

In some embodiments, the analytics may be related to at least one ofdescriptive analytics, diagnostic analytics, predictive analytics, orprescriptive analytics. The descriptive analytics describe what ishappening using compressive, accurate, and live data for effectivevisualization. The diagnostic analytics provide ability to drill down toa root cause and an ability to isolate confounding information. Thepredictive analytics describe business strategy, active adaption, andsimulation. The prescriptive analytics may be evidence-basedrecommendations to recommend actions and strategies based on challengertesting and may use advanced AI and analytical techniques to makespecific recommendations.

In some embodiments, the digital product network is a network ofproducts that have a familial relationship to each other. For example,the products may all have the same brand, be part of the same metaverse,generate data according to the same standardized format, be specificallydesigned to interact or communicate with each other, or the like. Thedigital product network collects information from different pieces of aproduct family to generate information and outcomes that are notavailable from a single product. The family may include physical goodsor physical goods mixed with content. For example, the content may betechnical data, user profile information, or other content.

In some embodiments, the analysis and manipulation may lead to valuableinsights throughout the development, manufacturing, supply chain, andcustomer relationship stages of a product lifecycle. For example,emerging connectivity and technologies in connected products enableintelligent provisioning, data aggregation, and analytics that can beused to create product connection, transaction, and enablementplatforms. The data generated by the connected products may be analyzedto bridge the gap between supply and demand chains.

Two of the types of interactions that may take place in the digitalproduct network are loosely coupled interactions and platforminteractions. For loosely coupled interactions, the products are notdirectly tied to each other. The products operate separately and do notinherently trust interactions with each other. For example, while carsare on the road, their systems do not rely on the data from other carsfor vehicle steering, acceleration, or braking control. Products inthese types of interactions may use shared intention information, suchas when a nearby car indicates that the nearby car will soon changelanes. Products receiving and interpreting the intention may considerthe intention when evaluating response patterns. Data from products thathave loosely coupled interactions may be fused or integrated for use inartificial intelligence systems, machine learning systems, roboticprocess automation systems, digital twins, and the like.

Platform based interactions involve different products that share thesame operating platform or ecosystem. This shared ecosystem is able tocoordinate different kinds of response patterns and encourage or demanda specific response pattern. For example, smoke detectors in a buildingmay be designed to operate independently, but by sharing a commonframework they may trigger alarms or even critical event responses(e.g., set off sprinklers). A smoke detector may also increase its levelof monitoring (e.g., frequency of sensor polling, frequency of datatransmission, data fidelity) during periods of heightened alert toenable capabilities that may be otherwise dormant for reduced batteryconsumption. In the smoke detector example, the smoke detector may becommunicatively coupled to a self-driving car system such that the caris redirected away from fire events. In some embodiments, a self-drivingfire truck may drive to the fire event in response to the smoke detectoralert. Such a self-responding fire truck may be beneficial where, forexample, a community does not have a full-time professional firedepartment, such as in some rural locations where firefighters mayrespond to the fire location without first retrieving the fire truck.

In FIG. 150 , an example of a digital product network service 14002communicates with and executes algorithms related to a plurality ofdigital entities, such as connected products or intelligent products.For example, the digital product network service 14002 may be a versionof the intelligence service 1IT00 that is adapted for the specificfunctions of the digital product network 14000 described below.

In some example embodiments, the connected product may be enabled with aset of capabilities such as data processing, networking, sensing,autonomous operation, intelligent agent, natural language processing,speech recognition, voice recognition, touch interfaces, remote control,self-organization, self-healing, process automation, computation,artificial intelligence, analog or digital sensors, cameras, soundprocessing systems, data storage, data integration, and/or variousInternet of Things (IoT) capabilities, among others. The connectedproduct may include a form of information technology. The connectedproduct may have a processor, computer random access memory, and acommunication module. The product may be considered a value chainnetwork entity that fits in a value chain network to provide productusage data.

In the example provided, a connected product 14010, an ad-hoc network14012, an ad-hoc network 14014, a local network 14016, and a localnetwork 14018 communicate with the digital product network service 14002directly or through a network 14019.

The connected products may be consumer products, industrial products, orother products that have digital components that may communicate with orat least partially include the digital product network service 14002.The connected products may be various entities within variousindustries. For example, each of the connected products may relate toapparel, electronics (general), computers and computer peripherals,chemicals (specialty), machinery, food processing, auto parts, steel,retail (online), retail (distributors), retail (special lines), retail(general), retail (grocery and food), electronics (consumer and office),farming/agriculture, food wholesalers, or healthcare products.

The connected product 14010 communicates directly with the digitalproduct network service. For example, the connected product 14010 mayinclude an antenna to send electromagnetic band communications directlyto an entity that hosts the digital product network service 14002. Insome embodiments, the digital product network service 14002 is at leastpartially hosted within the connected product 14010. For example, theconnected product 14010 may include sensors that send sensor datadirectly to various intelligence layer modules of the digital productnetwork service 14002 that are programmed into processors of theconnected product 14010.

The ad-hoc network 14012 includes a connected product 14020 and aconnected product 14022. The connected product 14020 and the connectedproduct 14022 communicate directly with each other and collectivelycommunicate with the digital product network service 14002.

The ad-hoc network 14014 includes a connected product 14024 and aconnected product 14026. The connected product 14024 and the connectedproduct 14026 communicate directly with each other and communicate withthe digital product network service 14002 through the network 14019.

The local network 14016 includes a gateway 14030, at least one sensorsystem 14032, at least one connected product 14034, and additional datasources 14036. In the example provided, the gateway 14030 communicateswith each of the sensor systems 14032, connected products 14034, andadditional data sources 14036. The gateway 14030 then communicatesdirectly with the digital product network service 14002. In someembodiments, the gateway 14030 hosts the digital product network service14002.

The local network 14018 includes a gateway 14040, at least one sensorsystem 14042, at least one connected product 14044, and at least oneadditional data source 14046. The gateway 14040 communicates with eachof the sensor systems 14042, connected products 14044, and additionaldata sources 14046. The gateway 14040 then communicates with the digitalproduct network service 14002 through the network 14019. For example,the gateway 14040 may be a router or home automation system hub.

Network 14019 may be any network for communicating data across largedistances. In the example provided, the network 14019 is the Internetaccessed through an internet service provider (ISP).

FIG. 151 illustrates an example of a connected product 14110. In theexample provided, the connected product 14110 includes a networkinterface 14112, at least one processor 14114, and at least one memory14116. In some embodiments, the connected products 14010, 14020, 14022,14024, 14026, 14034, and 14044 have configurations that are similar tothat of the connected product 14110.

The connected product 14110 includes at least one network interface14112, at least one processor 14114, and at least one memory 14116. Thenetwork interface 14112 includes one or more communication units thatcommunicate with a network (e.g., the Internet, a private network, andthe like).

The processor 14114 may be any kind of computational or processingdevice capable of executing program instructions, codes, binaryinstructions and the like, including a central processing unit (CPU), ageneral processing unit (GPU), a logic board, a chip (e.g., a graphicschip, a video processing chip, a data compression chip, or the like), achipset, a controller, a system-on-chip (e.g., an RF system on chip, anAI system on chip, a video processing system on chip, or others), anintegrated circuit, an application specific integrated circuit (ASIC), afield programmable gate array (FPGA), an approximate computingprocessor, a quantum computing processor, a parallel computingprocessor, a neural network processor, or other type of processor.

In the example provided, the processor 14114 includes a data collectionmodule 14120, a data reporting module 14122, a data analysis module14124, and intelligence services 14126. For example, the modules may bededicated electronic circuits or non-transitory computer code committedto a computer for instructing the processor to perform the algorithmcoded therein and described herein.

The data collection module 14120 instructs the connected product 14110to receive data from sensors and/or the network interface 14112 andcommit the received data to the memory 14116. The data reporting module14122 retrieves data stored in the memory 14116 or redirects data fromthe sensors or the network interface 14112 and transmits the data outfrom the network interface 14112 to a recipient entity. The dataanalysis module 14124 and the intelligence services 14126 performanalysis and execute artificial intelligence and/or machine learningalgorithms on the data.

The memory 14116 may be any type of non-transitory storage medium, suchas one or more of a CD-ROM, DVD, memory, hard disk, flash drive, RAM,ROM, cache, network-attached storage, server-based storage, and thelike. The memory 14116 stores methods, programs, codes, programinstructions or other type of instructions capable of being executed bythe processor 14114.

The memory 14116 stores data in various data structures. In the exampleprovided, the data structures include a usage data structure 14130, asensor data structure 14132, and a derived data structure 14134. Theusage data structure 14130 stores data related to the use of theconnected product 14110. For example, the usage data structure may storethe time, location, settings, and other details regarding when and howthe connected product has been used by a user. The sensor data structure14132 stores data collected from sensors related to the connectedproduct 14110. For example, the sensor data may be stored as quantizeddigital data corresponding to analog sensor signals generated fromsensors within the connected product 14110 or received at the networkinterface 14112. The derived data structure 14134 stores data derivedfrom the usage data and the sensor data. For example, the data analysismodule 14124 may compare usage frequency from the usage data withenvironmental temperature from the sensor data to determine a frequencyof use categorized by environmental temperature range to be stored inthe derived data structure 14134.

FIG. 152 illustrates a digital product network 14200. In the exampleprovided, a collection of digital products 14210 share product data14212 and enhanced product data 14214 with an intelligence layer 14220.For example, products (including goods and services) may create andtransmit data, such as product level data, to a communication layerwithin the value chain network technology stack and/or to an edge dataprocessing facility. This data may produce enhanced product level dataand may be combined with third party data for further processing,modeling or other adaptive or coordinated intelligence activity at theintelligence layer. This may include, but is not limited to, producingand/or simulating product and value chain use cases, the data for whichmay be utilized by products, product development processes, productdesign, and the like.

The digital products 14210 may include industrial products 14216,consumer products 14218, and other types of connected products that maycommunicate with the intelligence layer 14220. In some embodiments, thedigital products 14210 include at least one connected product 14110 ofFIG. 151 .

A third-party data system 14221 shares third party data 14222 with theintelligence layer 14220. For example, the third-party data systems14221 may share third party data 14222 related to any supporting dataused by the intelligence layer 14220 that is not already available inproduct data 14212.

The intelligence layer 14220 includes various modules that performanalysis on the product data 14212, the enhanced product data 14214, andthe third party data 14220. The analysis outputs configured data 14226to send to at least one user system 14228.

In the example provided, the intelligence layer 14220 includes a demandaggregation module 14230, a supply chain management module 14232, a newproduct development module 14234, a customer relationship managementmodule 14236, a product lifecycle management (PLM) module 14238, adigital twin module 14240, a synchronized planning module 14242, anintelligent procurement module 14244, and a dynamic fulfillment module14246. It should be appreciated that other embodiments may include othercombinations and types of modules without departing from the scope ofthe present disclosure.

The user systems 14228 may be third party systems that purchase theconfigured data 14226, may purchase analytics produced by the usersystem 14228, or may be systems owned by the same enterprise thatoperates the intelligence layer 14220.

In the example provided, the user systems 14228 include a demandaggregation system 14250, a supply chain management system 14252, a newproduct development system 14254, a customer relationship managementsystem 14256, a product lifecycle management (PLM) system 14258, adigital twin system 14260, a synchronized planning system 14262, anintelligent procurement system 14264, and a dynamic fulfillment system14266. Each of the user systems corresponds to a different use for theconfigured data and is generated by a corresponding module in theintelligence layer.

PLM system 14238 may provide accurate and up-to-date product informationaccessible throughout the value chain and product lifecycle. The PLMsystem may enable enhanced cross-function and cross-organizationalinvolvement in the design, collaborative innovation, design formanufacture/procurement, platform-based design philosophies, quickertime-to-market, and improved portfolio management.

New product development system 14234 is associated with developing andmanaging product and service value chains that are responsive tocustomer experience and are transformed by smart real-time data,advanced technologies, and agile innovation. The new product developmentsystems may contribute to improved design quality, increasedproductivity, and enhanced communication and visibility. For example,improvements may be realized in: simulations that benefit from virtualmodel processing in detail without spending resources to physically testdesign in a real-world environment; monitoring with up to date/real timemonitoring of user habits for future designs; ease of assembly design;design for ease of manufacturing/collection of parts that will form theproduct after assembly; design and conformance to specificationsproviding the fundamental basis for managing operations to producequality products; and end-to-end transparency, real-time root causeanalytics, and proactive resolution driven by customer connectivity forfaster problem resolution, problem prevention, customer satisfaction,performance, compliance verification, and avoided warranties. Additionalbenefits may include supply chain management with the application of theInternet of Things, the use of advanced robotics, and the application ofadvanced analytics of big data in supply chain management performanceand customer satisfaction. In addition to driving other user systems,many of the additional benefits may influence design decisions. Demandforecasting may be improved with predictive analytics to understand andpredict customer demand to optimize supply decisions by corporate supplychain and business management. Predictive procurement may forecast thefuture price trends, price fluctuations, future risks to manage and thepotentials required with the aid of a proper analysis based on previousprocurement data. Real time/up to date product management may improvecustomer engagement and extend product lifecycles. Firms (manufacturingfirms or 3rd Party contractors) may develop the capabilities to provideservices and solutions that supplement their traditional productofferings (e.g., equipment maintenance, data migration, data storage).

In customer relationship management system 14236, an integrated solutioncan combine customer profiles, interactions, and transaction informationfrom multiple applications to provide a view of customers with asolution that is equipped with industry-specific functionalities as wellas business intelligence capabilities. For example, the digital productnetwork may enable: remote diagnosis such that the subjects can beseparated by physical distance instead of the subject being co-locatedwith the person or system performing diagnostics; inter-machineconnectivity (M2) by enabling a sensor or meter to communicate theinformation it records to application software that can use it; andwarranty/repair with proactive and pre-emptive warranty management thatbecomes easier with an IoT based digital warranty management system. Thedigital product network may further improve: brand/product agility withreal time product monitoring and ability to quickly survey themarketplace leading to greater brand agility; digital product qualitywith end-to-end transparency, real-time root cause analytics, andproactive resolution driven by customer connectivity; simulation bygiving customer tools to virtual model process/use VR and AR in detailwithout spending resources to physically test design in a real-worldenvironment; fractional ownership usage and tracking and physical goodreal-world compliance to meet parameters of agreements; common platformand product architecture for a set of stable components that supportvariety and ability to evolve in a system by constraining the linkagesamong the other components; and design for consumption insights andmethods to improve how customers use/consume products.

FIG. 153 illustrates an example digital product network 14300 and FIG.154 illustrates a method 14400 of using the product level data accordingto some embodiments of the present disclosure. The digital productnetwork 14300 is similar to the digital product network 14000, wherelike numbers refer to like components.

Digital products 14310 are similar to digital products 14210. In theexample provided, the digital products include a set of digital productseach having a product processor, a product memory, and a product networkinterface. In some embodiments, one of the digital products 14310 is aproduct network control tower that has a control tower processor, acontrol tower memory, and a control tower network interface. In someembodiments, the product network control tower is a server or a productthat is not one of the digital products 14310. The product processor andthe control tower processor collectively include non-transitoryinstructions that program the digital product network system asdescribed herein. For example, the intelligence layer 14320 may bedistributed amongst the digital products 14310 and remote servers.

In some embodiments, the digital products 14310 include a display 14311.In some embodiments, the display 14311 is associated with the productnetwork control tower. The display 14311 presents images to a user ofthe display 14311. For example, the display 14311 may be a screen on amobile phone, a television, a projector, or the like.

In the example provided, the set of digital products and the productnetwork control tower have a set of microservices and a microservicesarchitecture. The digital products 14210 or the digital products 14310generate product level data at the product processor in task 14410 ofmethod 14400. The digital products 14210 or digital products 14310transmit the product level data from the product network interface intask 14412.

The product network control tower receives the product level data at thecontrol tower network interface in task 14414. In the example provided,the product processor and the control tower processor are furtherprogrammed to communicate based on a shared communication systemconfigured for facilitating communication of the product level data fromthe set of digital products amongst themselves and with the productcontrol tower. In some embodiments, the shared communication systemincludes an electromagnetic licensed to an enterprise who operates orowns the digital product network 14300. In some embodiments, thecommunication system includes a shared security protocol forcommunicating over shared electromagnetic bands, local area networks,Internet, 5G, and the like.

The digital product network 14300 encodes the product level data as aproduct level data structure configured to convey parameters indicatedby the product level data across the set of digital products in task14416. The digital product network 14300 writes the product level datastructure to at least one of the product memory and the control memoryin task 14418. The digital product network 14300 processes the productlevel data structure in task 14420. The digital product network 14300transmits the processed data to the user system in task 14422.

In some embodiments, the intelligence layer 14320 includes a graphicaluser interface (GUI) module 14340 and a proximity module 14342. The GUImodule may generate at least one user interface display for presentationon the display 14311. The GUI module 14340 may generate the parametersof at least one digitally enabled product of the set of digital productsin the at least one user interface display and may generate a proximitydisplay of proximal digital products of the set of digital products inthe at least one user interface display. In some embodiments, generatingthe proximity display includes generating the proximity display ofproximal products that are geographically proximate, where the digitalproduct network is further programmed to filter the proximal products byat least one of product type, product capability, or product brand. Insome embodiments, generating the proximity display includes generatingthe proximity display of proximal products that are proximate to one ofthe set of digital products by product type proximity, productcapability proximity, or product brand proximity.

In some embodiments, the intelligence layer 14320 includes a dataintegration module 14344. In some embodiments, the intelligence layer14320 includes an edge computation and edge intelligence module or edgemodule 14346 for edge distributed decision making among the set ofdigital products. In some embodiments, the edge module 14346 isconfigured for edge network bandwidth management between or out of theset of digital products.

In some embodiments, the intelligence layer 14320 includes a distributedledger system module 14348. In some embodiments, the distributed ledgersystem may be distributed exclusively within the digital products 14310.In some embodiments, the distributed ledger is a block chain ledger.

In some embodiments, the intelligence layer 14320 includes a qualitymanagement system having a product complaint module 14350 for capturingproduct complaints at the set of digital products. In some embodiments,the digital products 14310 detect complaints about other digitalproducts 14310. For example, a digital product may use machine vision orsound processing to identify dissatisfaction of a user while the user isusing a different digital product.

In some embodiments, the intelligence layer 14320 includes a productcondition module 14352. Product condition module 14352 may identify acondition of the set of digital products. Product condition module 14352may further encode the condition as one of the parameters of the productlevel data structure. Product condition module 14352 may further yettrack and/or monitor the condition across the set of digital products.For example, a bicycle manufacturer may monitor the condition of soldsmart bicycles to determine potential demand for repair parts or newbicycles. In another example, a rentable scooter company may monitor thecondition of the active scooters in the scooter fleet to budget forrepairs and replacement scooters.

In some embodiments, the intelligence layer 14320 includes a smartcontract module 14354 for enabling the creation of smart contracts basedon the product level data structure. In some embodiments, theintelligence layer 14320 configures the smart contracts based on aco-location-sensitive configuration of terms such that smart contractterms and conditions depend on proximity of a plurality of digitalproducts of the set of digital products.

In some embodiments, the intelligence layer 14320 includes a roboticprogram automation (RPA) module 14356. In some embodiments, the RPAmodule 14356 is configured to gamify an interaction based on whatdigital products are in the set of digital products. In someembodiments, the RPA module 14356 generates RPA processes based on useof a plurality of digital products of the set of digital products.

FIG. 155 illustrates an example of a digital product network system14508 where a data enhancement system 14510 receives data from digitalproducts 14512 for use by data user systems 14514. In some embodiments,the data enhancement includes at least one of data fusion and dataintegration to leverage cross-product data. The data enhancement system14510 may be part of the intelligence service or layer, part of adigital twin, part of a control tower, in distributed processors in theproducts, or in other suitable systems.

The digital products 14512 are similar to the connected products 14110described above. Each of the digital products 14512 provides data to thedata enhancement system 14510. In the example provided, three differentdigital products 14512 are sharing data with the data enhancement system14510. The data may be usage data for the product, sensor data collectedby the product, data retrieved from other products, data incorporatedfrom external sources, or other data obtained through other methods. Forexample, usage data may include a timestamp indicating when a productwas used, length of time data indicating how long the product is in use,data indicating what other products the product interacted with duringuse, and any other suitable usage data. Sensor data may includeenvironmental data, condition data, image data, sound data, and thelike.

The data user system 14514 may be any systems that use or generateenhanced data. In some embodiments, the data user systems incorporatethe data enhancement system 14510. For example, a machine learningsystem may train on various data streams from the products 14512 togenerate enhanced data. In the example provided, the data user systems14514 include AI/ML systems, Robotic Process Automation (RPA) systems,and digital twin systems.

In some embodiments, the digital product network system 14508 is aconstruction, home improvement, quality control, or similar system. Forexample, the products 14512 may be part of a family of tools. A laserlevel may provide leveling data input to a hand held self-levellingdrill for ensuring accurate drill positioning. The digital productnetwork may retrieve relevant specifications (e.g., for proper loadbearing) provided from a job control facility. A tool belt may indicatethe appropriate tool to use or access for the next task based on thespecification. A tool box with digital capabilities may then indicatethe appropriate drill bit and fastener type based on the material of thesubstrate to be drilled. The self-leveling drill may then retrieve thespecification to set rotation speed, turn on or off a hammer drillfunction, etc. Data generated by the digital product tools may then becombined for validation of proper execution according to a workmanshipspecification.

In some embodiments, the digital product network system 14508 may be anair quality system, an energy auditing system, or a similar system. Forexample, the digital products 14512 may be cleaning products, airheating and cooling products, air filter products, window statedetection products, or the like. A digital vacuum cleaner may detect theamount of dust and debris picked up during vacuum operations. The dataenhancement system 14510 may then then fuse the dust data with data froma digital heating/cooling and air filter product system and weather datafrom a third party. Data user system 14514 may then train an AI system,an ML system, an RPA system, or the like with the fused data to predictindoor air quality metrics and potential causes of poor indoor airquality.

In some embodiments, the digital product network system 14508 is a sleepquality system. For example, the digital products 14512 may include adigital bed product, a light switch product, a refrigerator door statedetector, or the like. The digital bed product may detect sleepduration, restlessness, and other sleep data. The light switch productmay indicate the status of lights to indicate the amount of ambientlight near the bed. The refrigerator door state detector may indicatewhat time the door was last opened before the user started sleeping inthe bed. The data enhancement system 14510 may then fuse the sleep datawith refrigerator use data and ambient light data. The data user system14514 may then train an AI system, an ML system, an RPA system, or thelike with the fused data to identify potential foods or eating behaviorthat may be contributing to poor sleep.

In some embodiments, the digital product network system 14508 is anelectrical circuit analysis system. For example, the digital products14512 may include a circuit breaker digital product, sensitiveelectronic products, or the like. The circuit breaker digital productmay generate circuit use data. The sensitive electronic products maygenerate performance data or input voltage data. The data enhancementsystem 14510 may then fuse the circuit use data with performance dataand voltage input data from various home digital products. The data usersystem 14514 may then train an AI system, an ML system, an RPA system,or the like with the fused data to map circuits in a building, recommenddifferent receptacles to use for sensitive electronics to avoidperformance issues from voltage drops on heavily loaded circuits, or thelike.

In some embodiments, the digital product network system 14508 is a childentertainment management system. For example, the digital products 14512may include digital children toys, televisions with viewing categorydata, or the like. The digital children toys may generate use data. Thetelevisions may generate data indicating the time and duration ofchildren show viewership. The data enhancement system 14510 may thenfuse the children toy use data with the children show viewership. Thedata user system 14514 may then train an AI system, an ML system, an RPAsystem, or the like with the fused data to identify the types andfeatures of toys that draw children away from the television, market newtoys, develop new toys, calibrate toy focus groups, or the like.

In some embodiments, the digital product network system 14508 is acuration system for relating digital entertainment content to augmentedreality. For example, the digital products 14512 may includetelevisions, augmented reality headwear, GPS locators, or the like. Thedata enhancement system 14510 may then fuse the television data withlocation information and landmark information. The data user system14514 may then train an AI system, an ML system, an RPA system, or thelike with the fused data to contextualize augmented reality depictionsbased on the substance of entertainment choices made on a playbackdevice. For example, a television show set in town X may be linked tophone and headset digital products to present—in augmentedreality—depictions, labels, markers, and other things that arethematically related to the substance of the television show while theuser is in town X. The labels and markers may identify buildings wherescenes occurred in the television show, such as indicating that “thisbuilding is where the Mafia Boss Z ran his operation.” In someembodiments, the system may present material related to botany. Forexample, augmented reality may curate indicators of flora-relatedenvironment features, etc. This could be used to curate a “tour” of alocation new to a user. In some embodiments, the system may suggest fiveroutes through a city when the user enters the city. The routes may besupplemented by augmented reality pertaining to user interests inferredfrom user media/entertainment consumption. The system may then offerrelated content on a user media player, television, book reader, phoneetc.

In some embodiments, the digital product network system 14508 is anexercise system. For example, the digital products 14512 may includetreadmills, exercise bicycles, stair mills, medicine balls with sensors,machine vision products, or the like. The exercise products may generaterecordings, summaries, and analysis of workouts across a range ofdevices. The data enhancement system 14510 may then fuse the datagenerated across the exercise equipment products. The data user system14514 may train an AI system, an ML system, an RPA system, or the likewith the fused data to guide a user to use equipment in a manner thatcomplements what the user has done on other equipment. The system maycoordinate with devices that understand which muscle groups, calories,etc. are implicated or used. For example, when a user rides ten miles ona bike but has not used a treadmill, weights, step counter, etc., thesystem may indicate that the user should perform some exercise onspecific equipment for a given duration based on the usage information.In some embodiments, the system may be used to monitor patients incardio rehab. In some embodiments, the system may monitor athletes forsports-specific enhancements, including based on training sets of databy elite athletes across their platforms.

In some embodiments, the digital product network system 14508 is acarbon footprint calculation system. For example, the digital products14512 may include personal devices or devices that detect actions thatcontribute to carbon release, such as cars, thermostats, appliances,food purchases (POS data), clothing, etc. The data enhancement system14510 may then fuse the data generated across the carbon footprintcalculation products. The data user system 14514 may then train an AIsystem, an ML system, an RPA system, or the like with the fused data tocreate a plan in line with a personal goal, family goal, business goal,regulatory requirement, or the like. In some embodiments, the planimplements device control to limit use of high-carbon release applianceswhen the footprint exceeds a threshold.

In some embodiments, the digital product network system 14508 is across-platform or cross-product reputation system. For example, thedigital products 14512 may include various digital products that arecapable of interacting with online communities. The data enhancementsystem 14510 may then fuse the data generated across the digitalproducts. The data user system 14514 may then train an AI system, an MLsystem, an RPA system, or the like with the fused data to authenticateupstanding digital citizens and identify digital bad actors. The systemmay identify cheating, poor behavior, poor sportsmanship, adultlanguage, or other indicators of potential activities that may conflictwith terms of use or rules in various online communities, forums,applications, and games. The system may use metaverse IDs, governmentIDs, credit reports, criminal reports, or the like. The information maybe shared across devices (e.g., personal computers, gaming devices, gameconsoles, gaming handhelds, mobile devices, wearables, virtual realityheadsets, etc.) and databases. Metaverse IDs may be tied in withgovernment IDs (e.g., state IDs, federal IDs, driver's licenses,passports, etc.) which may then also be tied to an individual's IDacross one or more product categories (e.g., an ID for a website, an IDfor an entity, a general metaverse ID), such that poor behavior,cheating, hacking, or the like is able to be flagged and/or punishedacross platforms. In some embodiments, the following types of behaviorsare tracked: posting hateful content/discussion on social media, usingcheating third-party programs in competitive environments and/or secureapplications (e.g., cheat programs in video games, DDoSingwebsites/apps/game servers, terms of service violations ofwebsites/apps/secure databases, money/transaction fraud through takingadvantage of system vulnerabilities or third party programs, etc.). TheIDs may be used to log into PCs, laptops, mobile devices, smart watches,smart devices, or the like. The IDs may be tied to a network gateway,cellular IDs, or other information further up the data stream from thedevice to prevent data from any device, IP address, user, or the likerelated to the flagged ID from participating in certain activities.Also, such traffic and interactions may be throttled, may be modified,subject to auditing, subject to real-time AI/ML monitoring, and thelike. Furthermore, AI/ML processes may be trained to identify cheats,ToS violations, poor behavior, poor sportsmanship, etc. AI chipsets maybe developed and implemented in devices for identifying such behaviors,programs, and the like. In some embodiments, the system includesincentive programs that provide rewards (e.g., NFTs) for good behavioracross products. The rewards and financial details may be embodied in adigital wallet.

In some embodiments, the digital product network system 14508 is apersonal health management system. The digital products 14512 may be afamily that includes an implantable/permanent medical device, a wearabledevice, a smart phone, an external treatment device, or other healthproducts. Implantable devices may generate data for tracking of bloodchemistry, blood pressure, immune response, other “lab” data, as well asinternal load bearing (such as to measure relative pressures on ajoint). Wearable devices may measure and generate data for generalizedhealth conditions, movement, activity, etc. Smart phone devices maymeasure and generate data for location and various user behaviorcharacteristics, including social engagement, affect, happiness, andsocial metrics. External treatment devices may generate data indicatingcompliance with medication, physical therapy, and other treatmentregimens. The data enhancement system 14510 may then fuse the datagenerated across the digital products and the data user system 14514 maythen train an AI system, an ML system, an RPA system, or the like withthe fused data to form a digital twin of the patient and the regimen forsimulation, diagnosis, adjustment of treatment, communications andcoaching, advance problem detection, etc. For example, the personalhealth management system may automate diagnosis, prescription, insurance(underwriting, making claims, auditing, payout, adjustment), treatment(medication, PT, surgery), long-term health care planning andrecommendations, recommendations for wellness improvement (exercisemodifications, social engagement), gamification of health-relatedbehaviors, and other health related tasks and fields. In someembodiments, the system may monitor sleep patterns, heart rate, bloodpressure, and other health parameters combined with data from a smartrefrigerator to recommend the food a person should be eating to improveoverall health.

In some embodiments, the digital product network system 14508 is anautomobile digital reality system. The digital products 14512 mayinclude automobiles, hearing devices, augmented reality (AR) devices,virtual reality (VR) devices, mixed reality (MR) devices, and the like.Such a family of products may revolve around a customer's car and lookto leverage data streams across various products that could link to thevehicle. For example, fusing driving data and in-vehicle passengerobservation with hearable data including noise, entertainment tracks,and spoken word content may give a holistic sense of the driver orpassenger. An in-ear device may be specifically designed to work in afamily of products with the car. AR/VR headsets may be equipped to helpa driver learn to drive a specific car based on the current status andconfiguration of the car as determined by the products, rather than justfor a type or model of car detached from the current configuration. AnAR/VR headset may be configured to allow users to play social videogames, such as games where a car owner may race their own car againstfriends in their own cars based on data indicating current tirecondition, fuel levels, locations, etc.

In some embodiments, the digital product network system 14508 is aproduct family that has elements configured to reside and operate in adigital wallet, in the metaverse, in AR/VR devices, in a vehicle, inindividual rooms of the home, at work, in a smart city, in nature, etc.

In some embodiments, the digital product network system 14508 is anindustrial system. The digital products 14512 may include machinesensors indicating the need for adjustments to the machine (e.g., a needfor increasing fan usage for a machine that is too hot, a need forlubrication, a need for materials in terms of manufacturing or packagingsystems, a need for fixing broken or missing rollers indicated by aconveyor belt sensor, or the like). The data enhancement system 14510may fuse sensor data within a warehouse for specific purposes such asenvironment (e.g., temperature, airflow, humidity, lighting, UV light,etc.). This fusing may relate to each of these types of data, or acombination of this data may be fused on a separate device such that theresults of this data from an analysis engine may be only outputted to amobile device based on thresholds (e.g., too cold—suggest heat, too muchhumidity—initiate dehumidifier or initiate humidifier when too dry, pulldown shades if too much UV light, open windows or turn on fans forincreased airflow etc.). This environment data may be fused with sensordata relating to manufacturing systems or systems for packaging devices.This data may be fused separately to an analysis engine providingresults of machine statuses. The data user system 14514 may then trainan AI system, an ML system, an RPA system, or the like with the fuseddata to create an analysis engine, recommendation engine, and/orautomation engine that outputs to a software application on the mobiledevice with results of a combination of machine sensing and environmentsensing to provide recommendations and/or automate systems to resolveenvironment issues or machine status issues. The system may also monitora combination of environment conditions with machine status conditionsfor determining optimal conditions for greatest output to automaticallyadjust the machine and environment conditions to provide the optimalconditions while keeping costs minimal.

In some embodiments, the digital product network system 14508 is asports equipment system. The digital products 14512 may include golfclubs and golf balls, baseball bats and baseballs, hockey sticks andhockey pucks, tennis racquets and tennis balls, bowling balls, and thelike. The various striking implements and balls may generate dataindicating how far, how straight, how on target the ball travels. Amachine vision system or sensors in the striking implement may generatedata about acceleration, angle, rotation, and other data about the swingof the striking implement. The data enhancement system 14510 may thenfuse the data generated across the sports equipment, and the data usersystem 14514 may train an AI system, an ML system, an RPA system, or thelike with the fused data to identify corrections to coach the user andimprove the swing. In some embodiments, the system fuses data from smartexercise machines and smart watches/wearables.

In some embodiments, the digital product network system 14508 is aphysical retail system. The digital products 14512 may include items tobe purchased, packaging of the items, a shopping cart, a smartphone, orthe like. The products may indicate the type of items added to the cart,the quantity of items added to the cart, the items on a shopping list ina smartphone, etc. The data enhancement system 14510 may fuse the data,and the data user system 14514 may train an AI system, an ML system, anRPA system, or the like with the fused data to make suggestions forcomplementary products, indicate the locations of the complementaryproducts in the store, offer incentives, or the like. For example, theincentives may include discounts, reward points, digital badges, or thelike.

In some embodiments, the digital product network system 14508 is acommercial lending risk management system. The digital products 14512may include products in a warehouse, packaging, environment sensors, orsimilar products. The products may generate data indicating a proximityof different products in the same warehouse or the presence of the sameor different products in different warehouses. The data enhancementsystem 14510 may fuse the data, and the data user system 14514 may trainan AI system, an ML system, an RPA system, or the like with the fuseddata to alert the lender of movements in inventories that may indicaterisks or non-compliance with loan (credit line) terms or put pricing ina different bracket or trigger extra fees. The system may also indicateextreme risks that government authorities could take interest in, suchas hazardous or explosive materials stored or moved in dangerousconditions.

In some embodiments, the digital product network system 14508 is a mediaconsumption recommendation system. The digital products 14512 mayinclude microphones, televisions, and the like. The products maygenerate data indicating the music a person listens to, the person'staste in the books, the person's interest in television and moviecontent. The data enhancement system 14510 may fuse the data, and thedata user system 14514 may train an AI system, an ML system, an RPAsystem, or the like with the fused data to recommend video games thatmay appeal to the person.

In some embodiments, the digital product network system 14508 is ahealth improvement system. The digital products 14512 may includenetworked exercise equipment, rowing machines, stationary bicycles, asmart appliance (e.g., refrigerator), wearables (e.g., smart ring orsmart watch), smart beds, and other products. The products generate datathat may be used to gain a better understanding of a user's overallhealth. The data enhancement system 14510 may fuse the data, and thedata user system 14514 may train an AI system, an ML system, an RPAsystem, or the like with the fused data to learn about the user's eatinghabits, exercise habits, and sleeping/sitting habits. The system maydetermine if the user is burning too many calories or eating the wrongfoods given their workout routines. The system may also track the user'ssleep patterns and determine whether the user is exercising and/oreating at the right time. The networked exercise equipment may be ownedby a user or may be owned by a gym. When the equipment is located at agym, the exercise devices may pair with the user's phone or wearabledevice to know who is on the equipment, how long they used it, and thelike. The smart refrigerator may include sensors and imaging devicesthat determine what the user is buying, what they are actually eating,and when they are eating it. In some embodiments, the smart refrigeratorincludes a profile of the user (e.g., family with kids, single,cohabitating but no kids, etc.) and/or a voice-controlled interface thatverifies who is eating, what they are cooking and the like. The data maytrain the AI system to determine the user's overall health profile. Thesystem may be configured to make recommendations to the user, such asbetter foods to eat, better times to eat, how much to exercise, whattimes to exercise, and the like.

In some embodiments, the digital product network system 14508 is aproduct maintenance system. The digital products 14512 may include anyproduct or machinery that has some level of connectivity and somecomponents of warranties/repair. The resulting data sets and cumulativedata layer can be analyzed and used for remote diagnosis, warrantypricing, repair pricing, offers for replacements, offers for upgrades,or the like. For remote diagnosis, the subject can be separated byphysical distance instead of the subject being co-located with theperson or system performing the diagnostics. For warranty and repairpricing, proactive and pre-emptive warranty management becomes easierwith an IoT based digital warranty management system.

In some embodiments, the digital product network system 14508 is a powerconsumption management system. The digital products 14512 may includethermostats, light switches, light bulbs, refrigerators, coffee makers,HVAC products, and the like. A power consumption monitor split out bybreaker at an electrical service panel box can be linked to varioushousehold appliances/circuits. In some embodiments, the products providesufficient data that the circuit panel box monitor may be omitted infavor of watching the usage patterns by product. A single circuit may bemonitored for understanding different appliance consumption patterns(fridge on/off, vacuum on/off, HVAC on/off/heat/cool/fan) and may be aparent-umbrella over other smart devices (thermostats, lightswitches/bulbs, etc.). The system may combine the data and performanalysis of usage patterns (fridge temperature setting, coffee makeron-time, thermostat/HVAC usage, etc.) for maintenance, behavioral usageguidance, etc. For example, an HVAC may monitor airflow by powerconsumption (accounting for temperature, humidity, etc.) forrecommending filter changes or behavioral suggestions to save energy.

In some embodiments, the digital product network system 14508 is a taskmanagement system. The digital products 14512 may include locationsensors, status sensors, task completion sensors, and the like. The dataenhancement system 14510 may fuse the data, and the data user system14514 may train an AI system, an ML system, an RPA system, or the likewith the fused data to coordinate tasks and/or services in a metaverse,in real life, or in both. For example, the system may use geolocation,couponing, logistics, etc. to determine which employees, family members,or other participants are in the best position to perform a task. Thebest position may be pre-programed or AI optimized for priorities suchas savings, timing, importance, or the like. The system may provide taskordering, task instruction, and coordination of the other participantsaccordingly. The system may operate based on tasks to be performed inreal life and in the metaverse work environments and may link themetaverse and real life activities. In some embodiments, the system mayput services out to bid. In some embodiments, the system provides ametaverse work environment and a market for services in the metaversefor metaverse workers and real-life workers to work collaboratively toget to a result.

In some embodiments, the digital product network system 14508 is apersonal health system. The digital products 14512 may include wearabledevices, eyeglasses, eyeglasses with liquid lenses, active clothing(heating, cooling, stress/strain, forces, etc.), recreational devices(GPS monitor devices, etc.), building and other environmental systems(temperature, humidity, lighting, etc.), automotive systems, and thelike. The products generate sensor or other data defined (micro ormacro) that impacts personal health. The data enhancement system 14510may fuse multiple data streams, including those outside of ecosystemsthat could be integrated to monitor and manage personal health. The datauser system 14514 may train an AI system, an ML system, an RPA system,or the like with the fused data to augment and validate certainanalytical models (GPS navigation algorithms, etc.). The system mayprovide active alerts, automated reporting that builds a personalprofile over time, suggested active measures, instructions to enact theactive measures (e.g., by activated clothing or other devices, etc.).The system may perform psychological evaluation, recommendations,referrals, identification of hazards such as temperature, UV exposure,pathogen presence, etc., accumulated exposure to carcinogens or otherthings that could lead to long-term illness, life-long medical analysis,activating clothing for temperature changes, massage, etc.

Futures Smart Contract

FIG. 156 illustrates a smart futures contract system 15000. The smartfutures contract system 15000 may relate to, for example, a set of smartcontracts associated with various value chain network entities (e.g.,goods with a intelligence features, packaging or containers withintelligence features, infrastructure or fixtures with intelligencefeatures, transport systems with intelligence features, planningsystems, etc.), such as ones that are configured to manage or mitigaterisk (such as by hedging for or providing improved outcomes in case ofvarious potentially adverse contingencies, such as shortages in supply,supply chain disruptions, changes in demand, changes in prices ofinputs, changes in market prices, and the like), to provide operationalefficiencies (such as by insuring availability of items based on plansor predictions), to improve returns (such as by obtaining inputs at morefavorable prices than would otherwise be available, and the like) and/orto provide other benefits, such as by engaging with futures markets(including various markets for options, futures, and the like involvingcommodities, equities, currencies, energy, and other items) that arerelevant to a set of items that are provided by or within the valuechain network. The items involved in the smart contracts may includegoods, services, and any blended product that includes components ofgoods and services of the various types described herein and in thedocuments incorporated herein by reference.

In some embodiments, robotic process automation may operate indemand-side planning to orchestrate futures contracts. For example, arobotic agent may perform a set of de-risking algorithms to configureterms and conditions for a set of smart futures contracts that setprices, delivery times, and delivery locations for a set of inputs(e.g., parts, components, fuel, materials, or many others) that will berequired in order to provide a planned set of inventory of an item, suchthat the set of smart futures contracts automatically execute to obtaincommitments for supply upon discovery of market conditions that satisfya set of parameters or conditions (such as pricing conditions) set inthe de-risking algorithms. The robotic agent may be trained on atraining set of data, such as a training set of interactions of a set ofexpert procurement professionals with a set of inputs, such as demandplanning inputs (e.g., demand forecasts, inventory forecasts, and thelike, demand elasticity curves, predictions of competitive behavior,supply chain predictions and many others), including contractsrecommended or engaged by such professionals under such conditions. Thismay include interactions of such professionals within enterprise demandplanning software suites. The agent may include or be trained tointeract with a set of demand planning models, such as models thatforecast demand factors, supply factors, pricing factors, and otherfactors, including anticipated equilibria between supply and demand, andones that generate estimates of appropriate inventory, recommendationsfor pricing, location and timing recommendations for supply and/ordistribution, and the like. In embodiments, the de-risking algorithmsmay include ones for reducing a variety of risks and contingencies,including the ones noted above, such as shortages in supply, supplychain disruptions, changes in demand, changes in prices of inputs, andchanges in market prices, as well as ones involving macro-economicfactors, geopolitical disruptions, disruptions due to weather andclimate, impacts of epidemics or pandemics, counterparty risks(including anti-money laundering risks, credit risks, risk of default,and many others), and the like. In embodiments, de-risking algorithmsmay include algorithms that seek to mitigate risks created by use ofother algorithms, such as ones that help identify various biases, whichmay include input bias (such as biases in training inputs, biases inmodels, biases due to incomplete or inaccurate data, and the like),biases in weighting, and others, as well as ones that identify wherealgorithmic performance is inferior to human performance (such as whereintelligence systems cannot effectively replicate some important elementof a human decision maker). In embodiments, such de-risking algorithmscan provide a set of recommendations for adjustments to smart futurescontracts and/or to the de-risking algorithms that are used to configuresmart futures contracts. In some embodiments, the smart futures contractsystem 15000 operates from, embodies, or integrates with a digital twin(e.g., supply chain digital twin or a general digital twin interface).

In some embodiments, the smart contract system 15000 configures and/orenters a set of smart future contracts with the futures system 15006based on conditions in a value chain network. In some embodiments, thesmart futures contract system 15000 acts on the value chain networkbased on conditions and prices in a set of futures markets.

In some embodiments, the smart futures contract system 15000 may be atleast partially incorporated in a product or product packaging to manageor mitigate risk. For example, if a product or product packaging isexposed to adverse environmental conditions, a smart futures contractmay be automatically configured as an option to acquire a set ofreplacement products, covering the contingency that the product hasincurred damage that will require replacement. This may occur, forexample, while the product/package is still in transport, such asdetermined by sensors on the product, the package, a transport vehicle,or proximal infrastructure, such as before it is possible or convenientto test a set of products well enough to determine whether replacementwill in fact be required. Configuration of such an option-type futurescontract may be based on a model or predictive artificial intelligencesystem (such as one generated by an algorithm that may be trained onhistorical data sets and other inputs) that provides a prediction as tothe probability that a product (or some subset thereof) will need to bereplaced based on known exposure data, as well as upon a prediction ofthe impact of the need for replacement (including the impact of delaysand/or reduced supply on pricing and other factors). In embodiments, thesmart futures contract may be configured with an appropriate duration ofoption to allow for determination of the actual extent of need forreplacement, an appropriate option price, and the like, such that therisk of a catastrophic loss is mitigated, while the likelihood of aprofitable outcome is maintained to the extent possible under thecircumstances. In embodiments, an option-type futures contract toacquire replacement goods may be paired with an automatically configuredset of futures contracts that mitigate the risk by setting terms andconditions for alternatives to replacement, such as a set of smartcontracts that offer refunds to customers, that offer alternatives goodsor services, that offer incentives to accept delayed goods, or the like.Such contracts may be configured using similar inputs, models andalgorithms to the ones used for other smart futures contracts describedherein.

In some embodiments, the smart futures contract system 15000renegotiates a set of future prices based on a current market state.Renegotiation may be performed by a set of robotic process automationagents or other artificial intelligence system, such as trained onhistorical data, on feedback from outcomes, and/or upon humaninteractions involved in contract negotiations. As one of many examples,upon recognition of a likely widespread supply chain disruption for aninput component for a set of goods, the system may offer to renegotiatefuture pricing of inputs (such as to ensure continuity of supply),future pricing of outputs (such as to reflect likely increases in marketprices), and other factors, which may be offered in a set of futuressmart contracts that embody offered terms and conditions ofrenegotiation.

In some embodiments, the smart futures contract system 15000 relates toor undertakes predictive procurement to forecast future price trends,price fluctuations, future risks to manage, and other elementspotentially required, optionally with the aid of an analysis, model, orthe like based on previous procurement data. This may include modelsthat account for weather, climate, geopolitical situations,epidemics/pandemics, counterparty behaviors, government behavior(including import and export regulations and their enforcement),traffic, congestion at ports, inventory levels of key components,availability and pricing of materials, and many other factors.

In the example provided, the smart futures contract system 15000includes at least one contracting entity 15001, at least one data source15002, at least one intelligence service 15004, at least one futuressystem 15006, and at least one distribution system 15008. Thecontracting entity 15001 is the entity of the value chain network thatowns, rents, leases, purchases, or otherwise controls the intelligenceservice 15004. For example, the contracting entity 15001 may be amanufacturer of goods who is interested in managing the risk of rawmaterial scarcity in future product cycles. In another example, thecontracting entity 15001 may be an apparel manufacturer who isinterested in early identification and price negotiation for fabrics,dies, designs, or the like that may become popular for the next fashionseason. In yet another example, the contracting entity 15001 may be anindustrial entity who monitors the status of components within variousmachines and places orders for future delivery of components that arepredicted to fail in the near future. In the industrial entity example,the contracting entity 15001 may configure the intelligence service15004 to compare future prices for purchasing new machines with futuresprices for selling non-failing components of the machine to entercontracts for “parting out” a machine rather than repairing when futuresprices indicate that parting out and purchasing a new machine is lesscostly than continued repair.

The data source 15002 generates data for use by the intelligence service15004. The data may be locally measured (such as by sensors or IoTdevices), retrieved from third parties, determined or enhanced by otherintelligence services, or gathered in any other way without departingfrom the scope of the present disclosure. For example, the data source15002 may generate product level data, customer level data, or data atother value chain levels. In the example provided, the data source 15002includes directly connected customers 15010, intelligent products 15012,and environment sensors 15014. The intelligent products 15012 andenvironmental sensors 15014 may provide sensor data associated with anymeasurable parameter, such as at least one of vibration, humidity,temperature, pressure, proximity, level, accelerometers, gyroscope,infrared sensors, optical sensors, MEMS, liquid lenses, shock, security,machine, product, pneumatic, conductive, state dependent frequencymonitor, ultrasonic, capacitance, or microwave. The data source 15002may generate data associated with any suitable topic or industry, suchas at least one of Internet of Things (IoT), social networks, socialmedia, automated agent behavior, business entity behavior, humanbehavior, data source outcomes, data source parameters, wearables,personal, financial, economic, credit score, environment, weather,labor, employment, census, crime, health, living, journalism/media,entertainment, location/motion, loyalty, reputation, real estate,reviews, marketing, food and drug, education, retail, transportation,biometric, travel, event, or customer activity.

The intelligence service 15004 may be a configured version of theintelligence service 1IT00. For example, the intelligence service 15004may be adapted to execute the specific functions of the smart futurescontract system 15000 described below. In the example provided, theintelligence service 15004 includes at least one data storage 15019, asmart contract service 15020, a demand aggregation service 15022, adigital wallet 15024, a risk determination service 15026, and a roboticprocess automation (RPA) service 15028.

The data storage 15019 may be any type of non-transitory storage medium,such as one or more of a CD-ROM, DVD, memory, hard disk, flash drive,RAM, ROM, cache, network-attached storage, server-based storage, and thelike. The data storage 15019 stores methods, programs, codes, programinstructions, or other type of instructions capable of being executed byprocessors of the intelligence service 15004.

The data storage 15019 stores the data in various data structures. Inthe example provided, the data storage 15019 includes a risk datastructure 15021 and a robotic process automation (RPA) data structure15022. The risk data structure 15021 stores risk tolerance information,risk identification, risk assessment, and other risk informationassociated with the contracting entity 15001. For example, the risk datastructure 15021 may store a price fluctuation tolerance and a maximumprice the contracting entity 15001 may be interested in paying forproducts in the future.

The RPA data structure 15022 stores the robotic process automationalgorithms executed by the RPA service 15028. The algorithms may usevarious artificial intelligence embodiments described throughout thisdisclosure and the documents incorporated by reference herein, such asneural networks of various types, various algorithms and expert systems,and others.

The smart contract service 15020 creates, modifies, and monitorsperformance of smart contracts based on data from the data sources15002, information received from the futures systems 15006, and analysisfrom other features of the intelligence service 15004. For example, thesmart contract service 15020 may detail the type of commodity, a numberof units (e.g., barrels of oil, bushels of wheat, ounces of gold, or thelike), a contract price to be paid for the commodity, the execution dateof the futures contract, and other suitable parameter values in a smartcontract governing a futures contract with respect to a commodity. Insome embodiments, the smart contract service 15020 may indicateparameter values corresponding to triggering actions, such as initiatinga certification process associated with the transaction, initiating areporting process associated with the transaction, configuring logisticsinformation associated with the transaction, reconfiguring of terms(e.g., premium rates, interest rates, contract price, delivery date,payment due date, and/or the like). It should be appreciated that thetypes of data that may be used to parameterize a smart contract maydiffer without departing from the scope of the present disclosure.

In some embodiments, the smart contract service 15020 may operateautonomously. For example, the smart contract service 15020 may operatewithout human intervention based on instructions from the RPA service15028 or based on other criteria provided by the contracting entity15001.

In some embodiments, the demand aggregation service 15022 may predictthat demand for a specific raw material used in the manufacture of aproduct may increase in the near future based on data from environmentsensors 15014. Based on the predicted demand increase, the smartcontract service 15020 may create a smart contract with the futuressystem 15006 for future delivery of the raw materials in anticipation ofa price increase due to the predicted demand increase and other datafrom data sources 15002 suggesting that production of the raw materialsmay not outpace the anticipated rise in demand.

The smart contract service 15020 receives evidence of completion of atask to trigger actions (e.g., payments, recordation, or the like) inresponse to completed tasks. In another example, the smart contractservice 15020 may monitor futures pricing and purchase or sell goods andservices based on risk tolerances indicated by the contracting entity.

The demand aggregation service 15022 aggregates demand across groups,locales, etc. The aggregation may include aggregating demand for atleast one of hypothetical products, hypothetical events, hypotheticalservices, or services related to the hypothetical products or events.The demand aggregation service 15022 monitors demand response acrossmultiple systems, such as how demand responds to changes in supply(e.g., scarcity effects), price changes, customization, pricing,advertising, etc.

In some embodiments, the demand aggregation service 15022 aggregatesinformation, orders, and/or commitments (optionally embodied in one ormore contracts, which may be smart contracts) for one or more products,categories, raw materials, components, logistics reservations,consumables, equipment, or the like. The demand aggregation may includecurrent demand for existing products and future demand for products thatare not yet available.

The digital wallet 15024 stores banking and finance information forpayments into and out of the futures system 15006. For example, thedigital wallet 15024 may store cryptocurrency information, bank balanceand routing information, and other information that may be used tocomplete transactions in the futures systems and distribution systems15008.

The risk determination service 15026 determines the risk associated withevents and conditions received by data sources 15002, demand aggregationservice 15022, and futures system 15006. For example, the riskdetermination service 15026 may supply data to the smart contractsservice 15020 indicating that the financial risk of waiting to purchaseraw materials as they are needed is greater than the risk of purchasingthe raw materials in the futures system 15006 at some time before theraw materials are needed for delivery when the raw materials are needed.In some embodiments, the risk determination service 15026 retrieves riskinformation—such as price fluctuation tolerance—from the risk data15021. With a low price fluctuation tolerance, the smart contractservice 15020 may execute a smart contract for future delivery of theraw materials as the futures price approaches the maximum price thecontracting entity 15001 is interested in paying. With a high pricefluctuation tolerance, the smart contract service 15020 may allow thefuture price to exceed the maximum price in the hope that the price maylater decrease.

The robotic process automation (RPA) service 15028 may facilitate, amongother things, computer automation of producing and validating smartcontracts between the contracting entity 15001 and the futures system15006. In some embodiments, the RPA service 15028 monitors humaninteraction with various systems to learn patterns and processesperformed by humans in performance of respective tasks. This may includeobservation of human actions that involve interactions with hardwareelements, with software interfaces, and with other elements.Observations may include field observations as humans perform realtasks, as well as observations of simulations or other activities inwhich a human performs an action with the explicit intent to provide atraining data set or input for the RPA system, such as where a humantags or labels a training data set with features that assist the RPAsystem in learning to recognize or classify features or objects, amongmany other examples.

In some embodiments, the RPA service 15028 may learn to perform certaintasks based on the learned patterns and processes, such that the tasksmay be performed by the RPA service 15028 in lieu or in support of ahuman decision maker. For example, the RPA service 15028 may identifythat a farmer typically reserves trucking services to transport a cropto a place of sale approximately two weeks before harvesting a crop. TheRPA service 15028 may further identify that the farmer performs anannual service on a harvesting machine and that the crop has grown to aconsistent height approximately three weeks before harvesting the crop.Based on detecting the annual service of the harvesting machine (e.g.,by data from the machine itself, by identifying fuel filters in a creditcard receipt, by machine vision identifying the service, etc.) or basedon identifying that the height of the crop has reached the consistentheight (e.g., by machine vision data from data source 15002), the RPAservice 15028 may query the logistics reservations system 15038. Basedon the response from the logistics reservations system 15038, the smartcontract 15020 may present reservation options to the farmer ornegotiate reservations on the farmer's behalf for trucking servicesthree weeks from the service or crop height determination.

The futures system 15006 may be any system in which goods and servicesto be delivered or performed in the future are bought and sold. Forexample, the futures system 15006 may involve forward contracts, stockexchange futures, options, various derivatives, and the like. The goodsand services to be delivered or performed may include real property,commodities, raw materials, finished goods, computation services, or anyother physical material or performable service that may be subject to anobligation to deliver or perform in the future. In the example provided,the futures system 15006 includes components futures 15030, materialsfutures 15032, consumables futures 15034, equipment futures 15036, andlogistics reservations 15038.

The components futures 15034 may relate to machinery parts, repair partsof goods, wearable parts of goods, upgrade parts for goods, parts to beassembled by a manufacturer into a new good, and other component types.In some embodiments, the components futures 15030 are associated withcircular economy systems. For example, the smart contract service 15020may perform circular economy optimization based on futures pricing ofgoods, such as components.

The raw materials futures 15032 may relate to material used to createother goods. For example, the raw materials 15032 may be copper, steel,iron, lithium, and the like. The consumables futures 15034 may relate toitems that are consumed when creating goods or performing services. Forexample, the consumables may include razor blades, coffee pods, singleuse batteries, pork bellies, and the like.

The equipment futures 15036 may relate to machinery, vehicles, and otherequipment. The logistics reservations futures 15038 may relate to futureservices for warehousing, transportation, and the like. For example, thelogistics reservations futures 15038 may include port dockingreservations, trucking reservations, warehouse space rental, canalpassage reservations, and the like.

The distribution system 15008 relates to at least one of picking,packing, moving, storing, warehousing, transporting or delivering of aset of items in a supply chain. In the example provided, the smartcontract service 15020 enters contracts for delivery, storage, and otherhandling of the items with the logistics reservations system 15038 inconcert with delivery dates and locations detailed in smart contractsfor future delivery, storage, and handling of physical items through thedistribution system 15008.

In some embodiments, a value chain may include an that intelligent agentsystem receives feedback from users regarding respective intelligentagents. For example, in some embodiments, a client application thatleverages an intelligent agent may provide an interface by which a usercan provide feedback regarding an action output by an intelligent agent.In embodiments, the user provides the feedback that identifies andcharacterizes any errors by the intelligent agent. In some of theseembodiments, a report may be generated (e.g., by the client applicationor the platform) that indicates the set of errors encountered by theuser. The report may be used to reconfigure/retrain the intelligentagent. In embodiments, the reconfiguring/retraining an intelligent agentmay include removing an input that is the source of the error,reconfiguring a set of nodes of the artificial intelligence system,reconfiguring a set of weights of the artificial intelligence system,reconfiguring a set of outputs of the artificial intelligence system,reconfiguring a processing flow within the artificial intelligencesystem (such as placing gates on a recurrent neural network to render ita gated RNN that balances learning with the need to diminish certaininputs in order to avoid exploding error problems), reengineering thetype of the artificial intelligence system (such as by modifying theneural network type among a convolutional neural network, a recurrentneural network, a feed forward neural network, a long-term/short-termmemory (LSTM) neural network, a self-organizing neural network, or manyother types and combinations), and/or augmenting the set of inputs tothe artificial intelligence system.

In embodiments, a library of neural network resources representingcombinations of neural network types that mimic or simulate neocortexactivities may be configured to allow selection and implementation ofmodules that replicate the combinations used by human experts toundertake various activities that are subjects of development ofintelligent agents, such as involving robotic process automation. Inembodiments, various neural network types from the library may beconfigured in series and/or in parallel configurations to representprocessing flows, which may be arranged to mimic or replicate flows ofprocessing in the brain, such as based on spatiotemporal imaging of thebrain when involved in the activity that is the subject of automation.In embodiments, an intelligent software agent for agent development maybe trained, such as using any of the training techniques describedherein, to select a set of neural network resource types, to arrange theneural network resource types according to a processing flow, toconfigure input data sources for the set of neural network resources,and/or to automatically deploy the set of neural network types onavailable computational resources to initiate training of the configuredset of neural network resources to perform a desired intelligentagent/automation workflows. In embodiments, the intelligent softwareagent used for agent development operates on an input data set ofspatiotemporal imaging data of a human brain, such as an expert who isperforming the workflows that is the subject of development of a furtherand uses the spatiotemporal imaging data to automatically select andconfigure the selection and arrangement of the set of neural networktypes to initiate learning. Thus, a system for developing an intelligentagent may be configured for (optionally automatic) selection of neuralnetwork types and/or arrangements based on spatiotemporal neocorticalactivity patterns of human users involved in workflows for which theagent is trained. Once developed, the resulting intelligentagent/process automation system may be trained as described throughoutthis disclosure.

In embodiments, a system for developing an intelligent agent (includingthe aforementioned agent for development of intelligent agents) may useinformation from brain imaging of human users to infer (optionallyautomatically) what data sources should be selected as inputs for anintelligent agent. For example, for processes where neocortex region O1is highly active (involving visual processing), visual inputs (such asavailable information from cameras, or visual representations ofinformation like price patterns, among many others) may be selected asfavorable data sources. Similarly, for processes involving region C3(involving storage and retrieval of facts), data sources providingreliable factual information (such as blockchain-based distributedledgers) may be selected. Thus, a system for developing an intelligentagent may be configured for (optionally automatic) selection of inputdata types and sources based on spatiotemporal neocortical activitypatterns of human users involved in workflows for which the agent istrained.

FIG. 157 illustrates an example environment of an edge networking (EDNW)system 16100 according to some embodiments of the present disclosure. Inembodiments, the edge networking system 16100 provides a framework forproviding edge networking services to one or more edge environments16102-108. In some embodiments, the edge networking service 16100 may beat least partially replicated in respective edge environments 16102-108.Examples of edge environments 16102-108 in which the edge networkingsystem 16100 may be at least partially replicated include devices 16102,premises 16104, telecom installations 16106, and computing clouds 16108.In these embodiments, an individual instance of the edge networkingsystem 16100 may include some or all of the capabilities and/or modulesof the edge networking system 16100 discussed herein, whereby the edgenetworking system 16100 is adapted for the specific functions performedby the respective edge environment 16102-108 on which the edgenetworking system 16100 is replicated. Examples of edge devices 16102include client devices, smart devices, and connected devices, wearabledevices, and any other suitable device. Examples of the premises 16104include home/residential buildings, factory buildings, office buildings,campuses, government buildings, medical buildings, and any othersuitable building, campus, or premises. Examples of edge telecominstallations 16106 include base stations, satellites, regional datacenters, relay stations, signal arrays, and any other suitable telecominstallation. Examples of computing clouds 16108 include in-cloudcomputation networks, private clouds, public clouds, hybrid clouds,multiclouds, infrastructure-as-a-service clouds, platforms-as-a-serviceclouds, software-as-a-service clouds, and any other suitable type ofcomputing clouds.

Additionally or alternatively, in some embodiments, the edge networkingsystem 16100 may be implemented as a set of microservices, such thatdifferent edge environments 16102-108 may leverage the edge networkingsystem via one or more APIs exposed to the edge environments 16102-108.In these embodiments, the edge networking system 16100 may be configuredto perform various types of edge networking services that may be adaptedfor different edge environments 16102-108. In either of theseconfigurations, an edge environment 16102-108 may provide a networkingrequest to the edge networking system 16100, whereby the request is toperform a specific networking task (e.g., a connection initiation, aseries of connection initiations, an encryption of data, a transmissionof encrypted data over a connection, usage of a protocol for connectionand/or data transmission, a routing determination or calculation, usageof an application-specific protocol for data transmission, an AI chipsetinterfacing instance, 5G software definition, AI-assisted or enablednetworking, tunable signal filtering, AI-assisted or enabled networkenhancement, or digital twin network formulation, simulation,prediction, testing, and/or the like). In response, the edge networkingsystem 16100 executes the requested networking task for the respectiveedge environment 16102-108.

Additionally or alternatively, in some embodiments, the edge networkingsystem 16100 may be implemented using one or more specialized chips thatare configured to microservices and/or networking tasks. In embodiments,the edge networking system 16100 may communicate via the VCN bus 16110.One or more of the edge environments 16102-108 may be connected to theVCN bus, thereby allowing instances of the edge networking system 16100to communicate with one another via the bus. The VCN control tower 16112may also be connected to the bus. As such, one or more instances of theedge networking system 16100 may transmit data to and receive signalsfrom the VCN control tower 16112.

FIG. 158 illustrates an exemplary embodiment of an environment of theedge networking system 16100 wherein a plurality of VCN system services16202-206 are connected to the VCN bus 16110. In embodiments, the edgenetworking system 16100 may transmit data to and/or receive data fromone or more of the VCN control tower and/or other VCN system services16202-206 (e.g., the DPNW system, the ROBO system, an energy system andprocess such as NRGY system, etc.). The energy system and process may bereferred to as the energy system (e.g., NRGY system) which may be or mayinclude an energy system, process, module, service, platform, and/or thelike as described in the disclosure. Data received by the edgenetworking system 16100 from the VCN control tower 16112 may include,for example, data related to VCN tasks to be directed by the VCN controltower and completed by sharing of data between multiple of the VCNentities. The edge networking system 16100 may perform one or morenetworking tasks according to the one or more VCN tasks, such asencryption codes, routing data, protocol information, data, AIpredictions, AI routing information, AI conclusions, or any othersuitable data. The edge networking system 16100 may additionally oralternatively transmit data to other of the VCN entities via the VCNbus.

In some embodiments, the edge networking system 16100 is configured tofacilitate optimization of communication and processes between other VCNmodules. The edge networking system 16100 may perform one or morenetworking tasks with relation to one or more of the other VCN modules,for example by optimizing data packet communication and/or encryptionprotocols for a data stream between two of the other VCN modules, and/orbetween one or more data sources and other of the VCN modules. Forexample, a VCN task requiring fast transmission of data from IoT sensorsto one or modules of the ROBO system as well as fast reporting of energysystems by the NRGY system to the ROBO system may involve the VCNcontrol tower 16112 instructing the edge networking system 16100 tooptimize data routes and protocols between the ROBO system and one ormore of the IoT data sources and the NRGY system.

FIG. 159 illustrates an exemplary embodiment of an edge device 16102containing an instance of the edge networking system 16100. While FIG.159 illustrates the edge networking system 16100 installed on a device16102, it is to be appreciated that the edge networking system and/orconfigured instances thereof may be uploaded to/installed/operated on orwithin any suitable type of edge environment 16102-108 disclosed herein.The instance of the edge networking system 16100 may include one or moremodules configured to facilitate performing of networking tasks by theedge networking system 16100. The modules that an instance of the edgenetworking system 16100 may be specifically changed, added, removed,and/or otherwise tailored to suit the desired performance of theinstance of the edge networking system 16100 for the particular edgeenvironment on which the configured instance of the edge networkingsystem 16100 is stored. For example, different configurations of theedge networking system 16100 may be configured for and uploadedto/installed on each of the different types of edge environments16102-108, as well as to/on individual edge environments within thetypes of edge environments 16102-108 on a case-by-case basis. Theconfigurations of the individual instances of the edge networking systemmay be determined by the VCN control tower 16112 according to one ormore of the VCN tasks.

In embodiments, the edge networking system 16100 may include one or moreof the following modules: edge device-as-a-service module 16302,application-specific protocol module 16304, edge robotics module 16306,SDWAN module 16308, network customization module 16310, AI chipsetinterfacing module 16312, 5G software definition module 16314, edgenetworking AI module 16316, tunable signal filtering module 16318,network routing module 16320, AI network enhancement module 16322, andedge network digital twin module 16324.

In embodiments, the edge device-as-a-service module 16302 is configuredto facilitate provision of one or more functions of the edge environment16102-108 on which the edge networking system 16100 is installed and/orof one or more functions of the configured edge networking device 16100to other devices and/or platforms with which the edge networking system16100 is in communication, such as via the VCN bus 16110. For example, aconfigured instance of the edge networking system 16100 installed on asmart container may make functions of the smart container available toother connected items (e.g., a fulfillment system) via the edgedevice-as-a-service module 16302.

In embodiments, the application-specific protocol module 16304 isconfigured to determine and/or facilitate provision of one or moreapplication-specific networking protocols to the edge environment16102-108 on which the edge networking system 16100 is installed and/orto one or more other devices and/or platforms with which the edgenetworking system 16100 is in communication, such as via the VCN bus16110. For example, the application-specific protocol module 16304 mayfacilitate communication to and/or from an application that requirestransmitting and/or receiving data via a proprietary encryptedcommunication protocol.

In embodiments, the edge robotics module 16306 is configured tofacilitate networking related to robotics at the edge environment16102-16108. The edge robotics module 16306 may, for example, provideone or more of robotics-related data routing, robotics-relatedcommunication protocol selection and enabling, local controlinterfacing, remote control interfacing, sensor data transmission andreception, and other suitable features to robots local to and/or remotefrom the edge environment 16102-108 on which the edge networking system16100 is installed.

In embodiments, the SDWAN module 16308 is configured to facilitatecreation, management, and/or handling of communications via asoftware-defined wide area network (SDWAN). The edge networking system16100 may, for example, define a SDWAN via the SDWAN module 16308,and/or some or all of the devices connected to the SDWAN may haveinstances of the edge networking system 16100 installed thereon tofacilitate communications within the SDWAN defined by one or more SDWANmodules 16308.

In embodiments, the network customization module 16310 is configured tofacilitate creation, management, customization, and/or handling of oneor more networks. The edge networking system 16100 may customize anetwork for a specific VCN task via the network customization module16310. For example, the network customization module may facilitatecustomization of a network related to a VCN task involving demandprediction for a particular product and/or industry. The network may becustomized to facilitate fast, efficient, secure communication betweendevices connected thereto.

In embodiments, the AI chipset interfacing module 16312 is configured toenable performing of one or more networking tasks by the edge networkingsystem 16100 via one or more AI chipsets. For example, the edgenetworking system 16100 may make one or more network routingdeterminations via an AI system embedded on an AI chipset. Additionallyor alternatively, the AI chipset interfacing module 16312 may beconfigured to facilitate receiving data from and/or transmitting data toan AI chipset in communication with the edge environment 16102-108 onwhich the edge networking system 16100 is installed, such as via the VCNbus 16110.

In embodiments, the 5G software definition module 16314 is configured tofacilitate management, customization, handling, and/or other tasksrelated to 5G networks. For example, telecom installation 16106 on whichthe edge networking system 16100 is installed may be or include a 5Gbase station, and the software definition module 16314 of the edgenetworking system 16100 of the 5G base station may perform one or morenetworking tasks related to managing traffic passing through the 5G basestation. Additionally or alternatively, in some examples, 5G-enabledmobile devices connected to a 5G network may have communication via thenetwork facilitated by instances of the 5G software definition module16314 installed thereon.

In embodiments, the edge networking AI module 16316 is configured toperform artificial intelligence and/or machine-learning-relatedfunctions related to one or more of the networking tasks. For example,routing determinations, protocol selections, protocol management,encryption processes, filtering determinations, and/or any othersuitable type of networking task may be performed and/or assisted byAI/ML via the edge networking AI module 16316.

In embodiments, the tunable signal filtering module 16318 is configuredto perform tuning of digital signals transmitted and/or received byand/or within the edge environment 16102-108. The tunable signalfiltering module 16318 may include one or more tunable digital filters.The tunable digital filters may tune signals to, for example, reducenetwork traffic by culling unnecessary communications, remove or reducethird-party signals “piggybacking” on the edge environment 16102-108,filter IoT sensor data, and/or the like.

In embodiments, the network routing module 16320 is configured toperform network routing operations with respect to signals sent and/orreceived by the edge environment 16102-108. The network routingoperations may include networking tasks such as, for example,determining an optimal data path for sensor data within a premises,determining optimal traffic flow for a 5G network, determining a routingdevice by which mobile data should be transmitted, and the like.

In embodiments, the AI network enhancement module 16322 is configured tooptimize network performance via one or more AI and/or machine-learningprocesses. For example, the AI network enhancement module 16322 may useone or more AI and/or machine learning processes to determine optimalprotocols for data throughput, and/or to make predictions and/orsimulations of network conditions and congestion/throughput thereof.

In embodiments, the edge network digital twin module 16324 is configuredto create and/or manage one or more digital twins related to networkingby, at, and/or on one or more of the edge environments 16102-108. Forexample, the digital twin module 16324 may model a network within aproduction factory and run simulations via the digital twin to predictnetwork congestion.

In some embodiments, the edge network system 16100 has a system fordecoupling congestion control from link loss.

In some embodiments, the edge network system 16100 has an intelligentlayer on top of UDP.

In some embodiments, the edge network system 16100 has an automatedpolicy and governance engine for edge workload deployment.

In some embodiments, the edge network system 16100 has edge dataintegration with service-oriented architecture.

In some embodiments, the edge network system 16100 has edge-specificprotocols.

In some embodiments, the edge network system 16100 has an edge storageprotocol.

In some embodiments, the edge network system 16100 has an edge storageprotocol integrated with AI-managed storage.

In some embodiments, the edge network system 16100 has a distributededge database.

In some embodiments, the edge network system 16100 has edge-distributedquery language.

In some embodiments, the edge network system 16100 has an edge policyengine.

In some embodiments, the edge network system 16100 has an automated edgedata marketplace.

In some embodiments, the edge network system 16100 has an RF filteringsystem for wireless nodes.

In some embodiments, the edge network system 16100 has network coding.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system at each node of anetwork.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system at each node of a networkconfigured for filtering and multi-level signal compression based onsignal characteristics.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system at each node of a networkconfigured for filtering based on context and/or content.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system at each node of a networkconfigured for filtering based on feedback on outcomes.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system configured for archivingat an optimal level of granularity.

In some embodiments, the edge network system 16100 has an AI-enhancededge-aware network fabric.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system configured foroptimization of storage capacity.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system configured foroptimization of compute.

In some embodiments, the edge network system 16100 has a machinelearning and/or artificial intelligence system configured foroptimization of energy.

In some embodiments, the edge network system 16100 has a mail protocolfor mail.

In some embodiments, the edge network system 16100 has a video streamingprotocol for video streaming.

In some embodiments, the edge network system 16100 has a gaming protocolfor gaming.

In some embodiments, the edge network system 16100 has a system formonitoring packet activity.

In some embodiments, the edge network system 16100 has a system forstoring and replicating data streams for later use by a machine learningand/or artificial intelligence system.

In some embodiments, the edge network system 16100 has robotics as edgedevices.

In some embodiments, the edge network system 16100 has a system forenabling local control.

In some embodiments, the edge network system 16100 has a system forcustomizing security of the network and/or devices on the network.

In some embodiments, the edge network system 16100 has a system forconfiguring channels for various types of communications.

In some embodiments, the edge network system 16100 has a system forconfiguring an environment after discovery of devices.

In some embodiments, the edge network system 16100 has a system forcustomizing a network by data scheduling and resource availability.

In some embodiments, the edge network system 16100 has a system forcustomization of data routing, processing, and/or computing based ondata type and/or security.

In some embodiments, the edge network system 16100 has customvirtualization deployments.

In some embodiments, the edge network system 16100 has a system forcustomization of neural network layering to form new AI structures.

In some embodiments, the edge network system 16100 has a system forcustomization of antenna layouts and power supply to meet specifiedneeds.

In some embodiments, the edge network system 16100 has a system forenabling wireless charging that includes AI-based battery optimizationand AI management of heat production.

In some embodiments, the edge network system 16100 has a system forcustomizing the human interface.

In some embodiments, the edge network system 16100 has a system forcustomization of the optimal routing algorithm in service customized 5Gnetworks.

In some embodiments, the edge network system 16100 has a network device.

In some embodiments, the edge network system 16100 has an AI chipset.

In some embodiments, the edge network system 16100 has channelizationbeyond the front end.

In some embodiments, the edge network system 16100 has channelizationbeyond the interface.

In some embodiments, the edge network system 16100 has a system forcreating and managing a digital twin of a network and/or AI edge system.

In some embodiments, the edge network system 16100 has anedge-device-as-a-service system.

In some embodiments, the edge network system 16100 has areader-as-a-service system.

In some embodiments, the edge network system 16100 has agateway-as-a-service system.

In some embodiments, the edge network system 16100 has arepeater-as-a-service system.

In some embodiments, the edge network system 16100 has an assettag-as-a-service system.

In some embodiments, the edge network system 16100 has a robotic datacollector-as-a-service system.

Quantum Computing for VCNS

FIG. 160 illustrates an example quantum computing system 17000 accordingto some embodiments of the present disclosure. In embodiments, thequantum computing system 17000 provides a framework for providing a setof quantum computing services to one or more quantum computing clients.In some embodiments, the quantum computing system 17000 framework may beat least partially replicated in respective quantum computing clients(e.g., VCN control towers and/or various VCN entities). In theseembodiments, an individual client may include some or all of thecapabilities of the quantum computing system 17000, whereby the quantumcomputing system 17000 is adapted for the specific functions performedby the subsystems of the quantum computing client. Additionally, oralternatively, in some embodiments, the quantum computing system 17000may be implemented as a set of microservices, such that differentquantum computing clients may leverage the quantum computing system17000 via one or more APIs exposed to the quantum computing clients. Inthese embodiments, the quantum computing system 17000 may be configuredto perform various types of quantum computing services that may beadapted for different quantum computing clients. In either of theseconfigurations, a quantum computing client may provide a request to thequantum computing system 17000, whereby the request is to perform aspecific task (e.g., an optimization). In response, the quantumcomputing system 17000 executes the requested task and returns aresponse to the quantum computing client.

Referring to FIG. 160 , in some embodiments, the quantum computingsystem 17000 may include a quantum adapted services library 17002, aquantum general services library 17004, a quantum data services library17006, a quantum computing engine library 17008, a quantum computingconfiguration service 17010, a quantum computing execution system 17012,and quantum computing API interface 17014.

In embodiments, the quantum computing engine library 17008 includesquantum computing engine configurations 17016 and quantum computingprocess modules 17018 based on various supported quantum models. Inembodiments, the quantum computing system 17000 may support manydifferent quantum models, including, but not limited to, the quantumcircuit model, quantum Turing machine, spintronic computing system (suchas using spin-orbit coupling to generate spin-polarized electronicstates in non-magnetic solids, such as ones using diamond materials),adiabatic quantum computer, one-way quantum computer, quantum annealing,and various quantum cellular automata. Under the quantum circuit model,quantum circuits may be based on the quantum bit, or “qubit”, which issomewhat analogous to the bit in classical computation. Qubits may be ina 1 or 0 quantum state or they may be in a superposition of the 1 and 0states. However, when qubits have measured the result of a measurement,qubits will always be in is always either a 1 or 0 quantum state. Theprobabilities related to these two outcomes depend on the quantum statethat the qubits were in immediately before the measurement. Computationis performed by manipulating qubits with quantum logic gates, which aresomewhat analogous to classical logic gates.

In embodiments, the quantum computing system 17000 may be physicallyimplemented using an analog approach or a digital approach. Analogapproaches may include, but are not limited to, quantum simulation,quantum annealing, and adiabatic quantum computation. In embodiments,digital quantum computers use quantum logic gates for computation. Bothanalog and digital approaches may use quantum bits, or qubits.

In embodiments, the quantum computing system 17000 includes a quantumannealing module 17020 wherein the quantum annealing module may beconfigured to find the global minimum or maximum of a given objectivefunction over a given set of candidate solutions (e.g., candidatestates) using quantum fluctuations. As used herein, quantum annealingmay refer to a meta-procedure for finding a procedure that identifies anabsolute minimum or maximum, such as a size, length, cost, time,distance or other measure, from within a possibly very large, butfinite, set of possible solutions using quantum fluctuation-basedcomputation instead of classical computation. The quantum annealingmodule 17020 may be leveraged for problems where the search space isdiscrete (e.g., combinatorial optimization problems) with many localminima, such as finding the ground state of a spin glass or thetraveling salesman problem.

In embodiments, the quantum annealing module 17020 starts from aquantum-mechanical superposition of all possible states (candidatestates) with equal weights. The quantum annealing module 17020 may thenevolve, such as following the time-dependent Schrödinger equation, anatural quantum-mechanical evolution of systems (e.g., physical systems,logical systems, or the like). In embodiments, the amplitudes of allcandidate states change, realizing quantum parallelism according to thetime-dependent strength of the transverse field, which causes quantumtunneling between states. If the rate of change of the transverse fieldis slow enough, the quantum annealing module 17020 may stay close to theground state of the instantaneous Hamiltonian. If the rate of change ofthe transverse field is accelerated, the quantum annealing module 17020may leave the ground state temporarily but produce a higher likelihoodof concluding in the ground state of the final problem energy state orHamiltonian.

In embodiments, the quantum computing system 17000 may includearbitrarily large numbers of qubits and may transport ions to spatiallydistinct locations in an array of ion traps, building large, entangledstates via photonically connected networks of remotely entangled ionchains.

In some implementations, the quantum computing system 17000 includes atrapped ion computer module 17022, which may be a quantum computer thatapplies trapped ions to solve complex problems. Trapped ion computermodule 17022 may have low quantum decoherence and may be able toconstruct large solution states. Ions, or charged atomic particles, maybe confined and suspended in free space using electromagnetic fields.Qubits are stored in stable electronic states of each ion, and quantuminformation may be transferred through the collective quantized motionof the ions in a shared trap (interacting through the Coulomb force).Lasers may be applied to induce coupling between the qubit states (forsingle-qubit operations) or coupling between the internal qubit statesand the external motional states (for entanglement between qubits).

In some embodiments, a traditional computer, including a processor,memory, and a graphical user interface (GUI), may be used for designing,compiling, and providing output from the execution and the quantumcomputing system 17000 may be used for executing the machine languageinstructions. In some embodiments, the quantum computing system 17000may be simulated by a computer program executed by the traditionalcomputer. In such embodiments, a superposition of states of the quantumcomputing system 17000 can be prepared based on input from the initialconditions. Since the initialization operation available in a quantumcomputer can only initialize a qubit to either the |0> or |1> state,initialization to a superposition of states is physically unrealistic.For simulation purposes, however, it may be useful to bypass theinitialization process and initialize the quantum computing service17000 directly.

In some embodiments, the quantum computing system 17000 provides variousquantum data services, including quantum input filtering, quantum outputfiltering, quantum application filtering, and a quantum database engine.

In embodiments, the quantum computing system 17000 may include a quantuminput filtering service 17024. In embodiments, quantum input filteringservice 17024 may be configured to select whether to run a model on thequantum computing system 17000 or to run the model on a classiccomputing system. In some embodiments, quantum input filtering service17024 may filter data for later modeling on a classic computer. Inembodiments, the quantum computing system 17000 may provide input totraditional compute platforms while filtering out unnecessaryinformation from flowing into distributed systems. In some embodiments,the system 17000 may trust through filtered specified experiences forintelligent agents.

In embodiments, a system in the system of systems may include model orsystem for automatically determining, based on a set of inputs, whetherto deploy quantum computational or quantum algorithmic resources to avalue chain network activity, whether to deploy traditionalcomputational resources and algorithms, or whether to apply a hybrid orcombination of them. In embodiments, inputs to a model or automationsystem may include demand information, supply information, energy costinformation, capital costs for computational resources, developmentcosts (such as for algorithms), energy costs, operational costs(including labor and other costs), performance information on availableresources (quantum and traditional), and any of the many other data setsthat may be used to simulate (such as using any of a wide variety ofsimulation techniques described herein and/or in the documentsincorporated herein by reference) and/or predict the difference inoutcome between a quantum-optimized result and a non-quantum-optimizedresult. A machine learned model (including in a DPANN system) may betrained, such as by deep learning on outcomes or by a data set fromhuman expert decisions, to determine what set of resources to deploygiven the input data for a given request. The model may itself bedeployed on quantum computational resources and/or may use quantumalgorithms, such as quantum annealing, to determine whether, where andwhen to use quantum systems, conventional systems, and/or hybrids orcombinations.

In some embodiments, the quantum computing system 17000 may include aquantum output filtering service 17026. In embodiments, the quantumoutput filtering service 17026 may be configured to select a solutionfrom solutions of multiple neural networks. For example, multiple neuralnetworks may be configured to generate solutions to a specific problemand the quantum output filtering service 17026 may select the bestsolution from the set of solutions.

In some embodiments, the quantum computing system 17000 connects anddirects a neural network development or selection process. In thisembodiment, the quantum computing system 17000 may directly program theweights of a neural network such that the neural network gives thedesired outputs. This quantum-programmed neural network may then operatewithout the oversight of the quantum computing system 17000 but willstill be operating within the expected parameters of the desiredcomputational engine.

In embodiments, the quantum computing system 17000 includes a quantumdatabase engine 17028. In embodiments, the quantum database engine 17028is configured with in-database quantum algorithm execution. Inembodiments, a quantum query language may be employed to query thequantum database engine 17028. In some embodiments, the quantum databaseengine may have an embedded policy engine 17030 for prioritizationand/or allocation of quantum workflows, including prioritization ofquery workloads, such as based on overall priority as well as thecomparative advantage of using quantum computing resources versusothers. In embodiments, quantum database engine 17028 may assist withthe recognition of entities across value chain networks by establishinga single identity for that is valid across interactions and touchpoints.The quantum database engine 17028 may be configured to performoptimization of data matching and intelligent traditional computeoptimization to match individual data elements. The quantum computingsystem 17000 may include a quantum data obfuscation system forobfuscating data.

The quantum computing system 17000 may include, but is not limited to,analog quantum computers, digital computers, and/or error-correctedquantum computers. Analog quantum computers may directly manipulate theinteractions between qubits without breaking these actions intoprimitive gate operations. In embodiments, quantum computers that mayrun analog machines include, but are not limited to, quantum annealers,adiabatic quantum computers, and direct quantum simulators. The digitalcomputers may operate by carrying out an algorithm of interest usingprimitive gate operations on physical qubits. Error-corrected quantumcomputers may refer to a version of gate-based quantum computers mademore robust through the deployment of quantum error correction (QEC),which enables noisy physical qubits to emulate stable logical qubits sothat the computer behaves reliably for any computation. Further, quantuminformation products may include, but are not limited to, computingpower, quantum predictions, quantum optimizations, and quantum decisionsupport.

In some embodiments, the quantum computing system 17000 is configured asan engine that may be used to optimize traditional computers, integratedata from multiple sources into a decision-making process, and the like.The data integration process may involve real-time capture andmanagement of interaction data by a wide range of tracking capabilities,both directly and indirectly related to value chain network activities.In embodiments, the quantum computing system 17000 may be configured toaccept cookies, email addresses and other contact data, social mediafeeds, news feeds, event and transaction log data (including transactionevents, network events, computational events, and many others), eventstreams, results of web crawling, distributed ledger information(including blockchain updates and state information), results fromdistributed or federated queries of data sources, streams of data fromchat rooms and discussion forums, and many others.

In embodiments, the quantum computing system 17000 includes a quantumregister having a plurality of qubits. Further, the quantum computingsystem 17000 may include a quantum control system for implementing thefundamental operations on each of the qubits in the quantum register anda control processor for coordinating the operations required.

In embodiments, the quantum computing system 17000 is configured tooptimize the pricing of smart container-based freight transportationservices. In embodiments, the quantum computing system 17000 may utilizequantum annealing to provide optimized freight transportation servicepricing. In embodiments, the quantum computing system 17000 may useq-bit based computational methods to optimize pricing.

In embodiments, the quantum computing system 17000 is configured tooptimize design or configuration features of value chain networkproducts, devices, vehicles, services, and the like. For example, thequantum computing system 17000 may be configured to optimize a productdesign, a smart container design, a robot design, a smart containerfleet configuration, a robotic fleet configuration, a liquid lensdesign, a data story configuration, and many others. Additionally, oralternatively, the quantum computing system 17000 is configured tooptimize the movement or routes of value chain network entities,including robot or robotic fleet routes, smart container or smartcontainer fleet routes, and the like.

In embodiments, the quantum computing system 17000 is configured toautomatically discover smart contract configuration opportunities.Automated discovery of smart contract configuration opportunities may bebased on published APIs to marketplaces and machine learning (e.g., byrobotic process automation (RPA) of stakeholder, asset, and transactiontypes.

In embodiments, quantum-established or other blockchain-based smartcontracts applications may include, but are not limited to, booking aset of robots from a robotic fleet, booking a smart container from asmart container fleet, executing transfer pricing agreements betweensubsidiaries, and the like. In embodiments, quantum-established or otherblockchain-enabled smart contracts enable frequent transactionsoccurring among a network of parties, and manual or duplicative tasksare performed by counterparties for each transaction. Thequantum-established or other blockchain acts as a shared database toprovide a secure, single source of truth, and smart contracts automateapprovals, calculations, and other transacting activities that are proneto lag and error. Smart contracts may use software code to automatetasks, and in some embodiments, this software code may include quantumcode that enables extremely sophisticated optimized results.

In embodiments, the quantum computing system 17000 or other system inthe system of systems may include a quantum-enabled or other riskidentification module that is configured to perform risk identificationand/or mitigation. The steps that may be taken by the riskidentification module may include, but are not limited to, riskidentification, impact assessment, and the like. In some embodiments,the risk identification module determines a risk type from a set of risktypes. In embodiments, risks may include, but are not limited to,preventable, strategic, and external risks. Preventable risks may referto risks that come from within and that can usually be managed on arule-based level, such as employing operational procedures monitoringand employee and manager guidance and instruction. Strategy risks mayrefer to those risks that are taken on voluntarily to achieve greaterrewards. External risks may refer to those risks that originate outsideand are not in the businesses' control (such as natural disasters).External risks are not preventable or desirable. In embodiments, therisk identification module can determine a predicted cost for anycategory of risk. The risk identification module may perform acalculation of current and potential impact on an overall risk profile.In embodiments, the risk identification module may determine theprobability and significance of certain events. Additionally, oralternatively, the risk identification module may be configured toanticipate events.

In some embodiments, the quantum computing system 17000 or other systemof the system 17000 is configured for accelerated sampling fromstochastic processes for risk analysis. In embodiments,quantum-simulated accelerated testing is initialized to hold acceleratedlife tests with constant-stress loadings, including accelerateddegradation tests and time-varying stress loadings.

In embodiments, the quantum computing system 17000 or other system ofthe system 17000 is configured for graph clustering analysis for anomalyand fraud detection.

In some embodiments, the quantum computing system 17000 includes aquantum prediction module, which is configured to generate predictions.Furthermore, the quantum prediction module may construct classicalprediction engines to further generate predictions, reducing the needfor ongoing quantum calculation costs, which, can be substantialcompared to traditional computers.

In embodiments, the quantum computing system 17000 may include a quantumprincipal component analysis (QPCA) algorithm that may process inputvector data if the covariance matrix of the data is efficientlyobtainable as a density matrix, under specific assumptions about thevectors given in the quantum mechanical form. It may be assumed that theuser has quantum access to the training vector data in a quantum memory.Further, it may be assumed that each training vector is stored in thequantum memory in terms of its difference from the class means. TheseQPCA can then be applied to provide for dimension reduction using thecalculational benefits of a quantum method.

In embodiments, the quantum computing system 17000 is configured forgraph clustering analysis for certified randomness for proof-of-stakeblockchains. Quantum cryptographic schemes may make use of quantummechanics in their designs, which enables such schemes to rely onpresumably unbreakable laws of physics for their security. The quantumcryptography schemes may be information-theoretically secure such thattheir security is not based on any non-fundamental assumptions. In thedesign of blockchain systems, information-theoretic security is notproven. Rather, classical blockchain technology typically relies onsecurity arguments that make assumptions about the limitations ofattackers' resources.

In embodiments, the quantum computing system 17000 is configured fordetecting adversarial systems, such as adversarial neural networks,including adversarial convolutional neural networks. For example, thequantum computing system 17000 or other system of the platform 17000 maybe configured to detect fake trading patterns.

In embodiments, the quantum computing system 17000 includes a quantumcontinual learning (QCL) system 17032, wherein the QCL system 17032learns continuously and adaptively about the external world, enablingthe autonomous incremental development of complex skills and knowledgeby updating a quantum model to account for different tasks and datadistributions. The QCL system 17032 operates on a realistic time scalewhere data and/or tasks become available only during operation. Previousquantum states can be superimposed into the quantum engine to providethe capacity for QCL. Because the QCL system 17032 is not constrained toa finite number of variables that can be processed deterministically, itcan continuously adapt to future states, producing a dynamic continuallearning capability. The QCL system 17032 may have applications wheredata distributions stay relatively static, but where data iscontinuously being received. For example, the QCL system 17032 may beused in quantum recommendation applications or quantum anomaly detectionsystems where data is continuously being received and where the quantummodel is continuously refined to provide for various outcomes,predictions, and the like. QCL enables asynchronous alternate trainingof tasks and only updates the quantum model on the real-time dataavailable from one or more streaming sources at a particular moment.

In embodiments, the QCL system 17032 operates in a complex environmentin which the target data keeps changing based on a hidden variable thatis not controlled. In embodiments, the QCL system 17032 can scale interms of intelligence while processing increasing amounts of data andwhile maintaining a realistic number of quantum states. The QCL system17032 applies quantum methods to drastically reduce the requirement forstorage of historic data while allowing the execution of continuouscomputations to provide for detail-driven optimal results. Inembodiments, a QCL system 17032 is configured for unsupervised streamingperception data since it continually updates the quantum model with newavailable data.

In embodiments, QCL system 17032 enables multi-modal-multi-task quantumlearning. The QCL system 17032 is not constrained to a single stream ofperception data but allows for many streams of perception data fromdifferent sensors and input modalities. In embodiments, the QCL system17032 can solve multiple tasks by duplicating the quantum state andexecuting computations on the duplicate quantum environment. A keyadvantage to QCL is that the quantum model does not need to be retrainedon historic data, as the superposition state holds information relatingto all prior inputs. Multi-modal and multi-task quantum learning enhancequantum optimization since it endows quantum machines with reasoningskills through the application of vast amounts of state information.

In embodiments, the quantum computing system 17000 supports quantumsuperposition, or the ability of a set of states to be overlaid into asingle quantum environment.

In embodiments, the quantum computing system 17000 supports quantumteleportation. For example, information may be passed between photons onchipsets even if the photons are not physically linked.

In embodiments, the quantum computing system 17000 may include a quantumtransfer pricing system. Quantum transfer pricing allows for theestablishment of prices for the goods and/or services exchanged betweensubsidiaries, affiliates, or commonly controlled companies that are partof a larger enterprise and may be used to provide tax savings forcorporations. The quantum transfer pricing system is configured to solvea transfer pricing problem across all of the systems in the system ofsystems and the interfaces connecting the systems using quantumcomputing techniques. In embodiments, solving a transfer pricing probleminvolves testing the elasticities of each system in the system ofsystems with a set of tests. In these embodiments, the testing may bedone in periodic batches and then may be iterated. As described herein,transfer pricing may refer to the price that one division in a companycharges another division in that company for goods and services. Inembodiments, the quantum transfer pricing system may be applied across avalue chain network to optimize the overall product value.

In embodiments, the quantum transfer pricing system consolidates allfinancial data related to transfer pricing on an ongoing basisthroughout the year for all entities of an organization wherein theconsolidation involves applying quantum entanglement to overlay datainto a single quantum state. In embodiments, the financial data mayinclude profit data, loss data, data from intercompany invoices(potentially including quantities and prices), and the like.

In embodiments, the quantum transfer pricing system may interface with areporting system that reports segmented profit and loss, transactionmatrices, tax optimization results, and the like based on superpositiondata. In embodiments, the quantum transfer pricing system automaticallygenerates forecast calculations and assesses the expected local profitsfor any set of quantum states.

In embodiments, the quantum transfer pricing system may integrate with asimulation system for performing simulations. Suggested optimal valuesfor new product prices can be discussed cross-border via integratedquantum workflows and quantum teleportation communicated states.

In embodiments, quantum transfer pricing may be used to proactivelycontrol the distribution of profits within a multi-national enterprise(MNE), for example, during the course of a calendar year, enabling theentities to achieve arms-length profit ranges for each type oftransaction.

In embodiments, the QCL system 17032 may use a number of methods tocalculate quantum transfer pricing, including the quantum comparableuncontrolled price (QCUP) method, the quantum cost plus percent method(QCPM), the quantum resale price method (QRPM), the quantum transactionnet margin method (QTNM), and the quantum profit-split method.

The QCUP method may apply quantum calculations to find comparabletransactions made between related and unrelated organizations,potentially through the sharing of quantum superposition data. Bycomparing the price of goods and/or services in an intercompanytransaction with the price used by independent parties through theapplication of a quantum comparison engine, a benchmark price may bedetermined.

The QCPM method may compare the gross profit to the cost of sales, thusmeasuring the cost-plus mark-up (the actual profit earned from theproducts). Once this mark-up is determined, it should be equal to what athird party would make for a comparable transaction in a comparablecontext with similar external market conditions. In embodiments, thequantum engine may simulate the external market conditions.

The QRPM method looks at groups of transactions rather than individualtransactions and is based on the gross margin or difference between theprice at which a product is purchased and the price at which it is soldto a third party. In embodiments, the quantum engine may be applied tocalculate the price differences and to record the transactions in thesuperposition system.

The QTNM method is based on the net profit of a controlled transactionrather than comparable external market pricing. The calculation of thenet profit is accomplished through a quantum engine that can consider awide variety of factors and solve optimally for the product price. Thenet profit may then be compared with the net profit of independententerprises, potentially using quantum teleportation.

The quantum profit-split method may be used when two related companieswork on the same business venture, but separately. In theseapplications, the quantum transfer pricing is based on profit. Thequantum profit-split method applies quantum calculations to determinehow the profit associated with a particular transaction would have beendivided between the independent parties involved.

In embodiments, the system of systems may support quantum-aware devicestacks, including quantum-aware device-level kits, quantum-awareindustrial Internet of Things (IoT) kits, quantum-enabled FPGAs, andsystems with awareness of capabilities of different quantum computertypes and/or different quantum algorithm types.

In embodiments, the quantum computing system 17000 may leverage one orartificial networks to fulfill the request of a quantum computingclient. For example, the quantum computing system 17000 may leverage aset of artificial neural networks to identify patterns in images (e.g.,using image data from a liquid lens system), perform binary matrixfactorization, perform topical content targeting, performsimilarity-based clustering, perform collaborative filtering, performopportunity mining, or the like.

In embodiments, the system of systems may include a hybrid computingallocation system for prioritization and allocation of quantum computingresources and traditional computing resources. In embodiments, theprioritization and allocation of quantum computing resources andtraditional computing resources may be measure-based (e.g., measuringthe extent of the advantage of the quantum resource relative to otheravailable resources), cost-based, optimality-based, speed-based,impact-based, or the like. In some embodiments the hybrid computingallocation system is configured to perform time-division multiplexingbetween the quantum computing system 17000 and a traditional computingsystem. In embodiments, the hybrid computing allocation system mayautomatically track and report on the allocation of computationalresources, the availability of computational resources, the cost ofcomputational resources, and the like.

In embodiments, the quantum computing system 17000 may be leveraged forqueue optimization for utilization of quantum computing resources,including context-based queue optimizations.

In embodiments, the quantum computing system 17000 may supportquantum-computation-aware location-based data caching.

In embodiments, the quantum computing system 17000 may be leveraged foroptimization of various system resources in the system of systems,including the optimization of quantum computing resources, traditionalcomputing resources, energy resources, human resources, robotic fleetresources, smart container fleet resources, I/O bandwidth, storageresources, network bandwidth, attention resources, or the like.

The quantum computing system 17000 may be implemented in the system ofsystems architecture similarly to the intelligence service 17034, wherea complete range of capabilities are available to or as part of anyconfigured service. Configured quantum computing services may beconfigured with subsets of these capabilities to perform specificpredefined function, produce newly defined functions, or variouscombinations of both.

FIG. 161 illustrates quantum computing service request handlingaccording to some embodiments of the present disclosure. A directedquantum computing request 17102 may come from one or more quantum-awaredevices or stack of devices, where the request is for known applicationconfigured with specific quantum instance(s), quantum computingengine(s), or other quantum computing resources, and where dataassociated with the request may be preprocessed or otherwise optimizedfor use with quantum computing.

A general quantum computing request 17104 may come from any system inthe system of systems or configured service, where the requestor hasdetermined that quantum computing resources may provide additional valueor other improved outcomes. Improved outcomes may also be suggested bythe quantum computing service in association with some form ofmonitoring and analysis. For a general quantum computing request 17104,input data may not be structured or formatted as necessary for quantumcomputing.

In embodiments, external data requests 17106 may include any availabledata that may be necessary for training new quantum instances. Thesources of such requests could be public data, sensors, ERP systems, andmany others.

Incoming operating requests and associated data may be analyzed using astandardized approach that identifies one or more possible sets of knownquantum instances, quantum computing engines, or other quantum computingresources that may be applied to perform the requested operation(s).Potential existing sets may be identified in the quantum set library17108.

In embodiments, the quantum computing system 17000 includes a quantumcomputing configuration service 17010. The quantum computingconfiguration service may work alone or with the intelligence service17034 to select a best available configuration using a resource andpriority analysis that also includes the priority of the requestor. Thequantum computing configuration service may provide a solution (YES) ordetermine that a new configuration is required (NO).

In one example, the requested set of quantum computing services may notexist in the quantum set library 17108. In this example, one or more newquantum instances must be developed (trained) using available data. Forexample, a quantum computing module for optimizing truck freightdeliveries in the United States may exist in the quantum set library17108. However, requestor inputs identified the need to optimizeshipping in Canada. In this case, quantum instance training may workwith the intelligence service 17034 to train a new instance for Canadausing a range of public data such as shipping schedules, speed limits,fuel mileage and cost, and so forth. In embodiments, alternateconfigurations may be developed with assistance from the intelligenceservice 17034 to identify alternate ways to provide all or some of therequested quantum computing services until appropriate resources becomeavailable. For example, a quantum/traditional hybrid model may bepossible that provides the requested service, but at a slower rate.

In embodiments, alternate configurations may be developed withassistance from the intelligence service 17034 to identify alternate andpossibly temporary ways to provide all or some of the requested quantumcomputing services. For example, a hybrid quantum/traditional model maybe possible that provides the requested service, but at a slower rate.This may also include a feedback learning loop to adjust services inreal time or to improved stored library elements.

When a quantum computing configuration has been identified andavailable, it is allocated and programmed for execution and delivery ofone or more quantum states (solutions).

Biology-Based Systems for VCNS

Techniques described herein improve the ability of networks and systemsto collect, transmit, and process large volumes of data, especially datafrom sensors and other value chain data generators. These techniquesinclude using a thalamus service that provides an equivalent to abiological thalamus, a neural system for filtering and relaying data.The thalamus service described herein can receive large volumes ofinformation and quickly prioritize the information, passing on the mostimporting information so that limited transmission, processing,collection, and/or analysis resources are not overwhelmed by volume ofincoming information.

Additionally, a predictive model communication protocol (PMCP) isdescribed herein. PMCP may be used to reduce a volume of transmitteddata, especially when the data is predictable or usually predictable.PMCP may operate by transmitting predictive model parameters instead ofsome or all of the data values that would normally be transmitted by asensor device or other data source. For example, a device implementingPMCP may continually receive inputs (e.g., sensor data) and train apredictive model using the stream of sensor data. Rather thantransmitting the sensor data, which may use significant network and/orprocessing resources, the PMCP device may transmit the model parameters,which may be used by a receiving device to operate a predictive model topredict current and future sensor data. Thus, the receiving device mayhave a predictive model of sensor data without receiving the sensordata. In embodiments, if the sensor data at the PMCP device beginsoperating outside of expectations, the model parameters may bere-transmitted to the receiving device, which may update its predictivemodel and thereby obtain more accurate predictive data.

In some embodiments, to optimize decision-making, quantum computersand/or predictive models may be used with the techniques describedherein. Furthermore, quantum coordination can be applied to allow fordisparate units to securely coordinate actions (e.g., without the needfor traditional communication mechanisms). Accordingly, techniquesdescribed herein may use a combination of decentralized biology-baseddecision-making capabilities distributed throughout devices within thevalue chain network and quantum capabilities. Furthermore, thesetechniques are also an efficient mechanism for enabling operationalefficiency of a coordinated value chain network.

FIG. 162 shows a value chain network thalamus service 18000 and a set ofinput sensors streaming data from various sources across the value chainnetwork and the system of systems (SOS) control system 18002 with itscentrally-managed data sources 18004. The thalamus service 18000 filtersthe inputs from the various data sources 18004 into the control system18002 such that the control system is never overwhelmed by the totalvolume of information. In embodiments, the thalamus service 18000provides an information suppression mechanism for information flowswithin the value chain. This mechanism monitors all data streams andsuppresses and/or filters irrelevant data streams by ensuring that themaximum data flows from all input sensors are always constrained.

In embodiments, the thalamus service 18000 may be a gateway for allcommunication that responds to the prioritization of the system ofsystems control system 18002. The system of systems control system 18002may decide to change the prioritization of the data streamed from thethalamus service 18000, for example, during a known fire in an isolatedarea, and the event may direct the thalamus service 18000 to continue toprovide flame sensor information despite the fact that majority of thisdata is not unusual. The thalamus service 18000 may be an integral partof the overall system of systems communication framework.

In embodiments, the thalamus service 18000 includes an intake managementsystem 18006. The intake management system 18006 may be configured toreceive and process multiple large datasets by converting them into datastreams that are sized and organized for subsequent use by a centralcontrol system 18002 operating within a system of systems. For example,a robot may include vision and sensing systems that are used by thecentral control system 18002 (which may be on-board the robot and/or ina separate device in communication with the robot) to identify and movethrough an environment in real-time. The intake management system 18006can facilitate robot decision-making by parsing, filtering, classifying,or otherwise reducing the size and increasing the utility of multiplelarge datasets that would otherwise overwhelm the central control system18002. In embodiments, the intake management system may include anintake controller 18008 that works with the intelligence service 18010to evaluate incoming data and take actions-based evaluation results.Evaluations and actions may include specific instruction sets receivedby the thalamus service 18000, for example, the use of a set of specificcompression and prioritization tools stipulated within a “Networking”library module. In another example, thalamus service inputs may directthe use of specific filtering and suppression techniques. In a thirdexample, thalamus service inputs may stipulate data filtering associatedwith an area of interest such as a certain type of financialtransaction. The intake management system is also configured torecognize and manage datasets that are in a vectorized format such as inaccordance with a predictive model communication protocol (PMCP)(discussed below), where the datasets may be passed directly to thecentral control system 18002, or alternatively deconstructed andprocessed separately. The intake management system 18006 may include alearning module that receives data from external sources that enablesimprovement and creation of application and data management librarymodules. In some cases, the intake management system 18006 may requestexternal data to augment existing datasets.

In some embodiments, the SOS control system 18002 may direct thethalamus service 18000 to alter its filtering to provide more input froma set of specific sources. This indication to provide more input ishandled by the thalamus service 18000. For example, the thalamus servicemay suppress other information flows to constrain the total data flowsto within a volume that the central control system can handle.

In embodiments, the thalamus service 18000 can operate by suppressingdata based on several different factors including zero or more defaultfactors. For example, in some embodiments, the default factors mayinclude an “unusualness factor” that may be a value that indicates adivergence or a degree of divergence of the data from an expecteddataset. In embodiments, the unusualness factor is constantly monitoredfor all inputs or some of the inputs (e.g., some of the input sensors).

In some embodiments, the thalamus service 18000 may suppress data basedon geospatial factors. Examples of geospatial factors may includelocation data, motion data, acceleration data, vibration data, and/orany other data indicating an absolute or relative location, change inlocation over time, other derivatives or integrals of location overtime, etc. The thalamus service 18000 may be aware of the geospatialfactors for some or all of the sensors and thus is able to look forunusual patterns in data based on geospatial context and suppress dataaccordingly.

In some embodiments, the thalamus service 18000 may suppress data basedon temporal factors. Data can be suppressed temporally, for example, ifthe cadence of the data can be reduced such that the overall data streamis filtered to a level that can be handled by the SOS control system18002 and/or a central processing unit.

In some embodiments, the thalamus service 18000 may suppress data basedon contextual factors. In embodiments, context-based filtering is afiltering event in which the thalamus service 18000 is aware of somecontext-based event. Context-based events, for example, may include oneor more notifications of unusual behavior by other sensors or systems(which may lead to temporary suppression of less important data), one ormore human inputs (e.g., a human disabling a security alert, which maysuppress a previous focus on security data), one or more eventstriggered by other systems or sensors (e.g., an automated securityalert, which may lead to suppression of certain data to allow resourcesto be dedicated to security data collection, transmission, andanalysis), one or more contexts detected from other sensor data (e.g., areduction in available bandwidth reported by a network sensor, which maylead to the suppression of certain data until available bandwidthimproves), or any other context-based condition or event. In thiscontext, the filtering may suppress information flows not relating tothe data from the event.

In embodiments, the thalamus service 18000 may receive data from avariety of data sources 18004, including analyses 18018, databases18020, sensors 18022, and/or reports 18024. For example, the thalamusservice 18000 may receive analyses and/or reports from otheranalysis/processing/reporting devices that have already pre-processedsensor data or other data. Additionally or alternatively, the thalamusservice 18000 may receive data (e.g., historical data) that is stored ina database 18020 in addition to current or historical data from sensors18022. In embodiments, data may be received and/or generated (e.g.,predictive models may generate future data) from the PMCP deviceinterface 18052.

In embodiments, the thalamus service 18000 may process and/or interpretinputs from any of the data sources 18004 based on an intake applicationlibrary 18012, which may include a networking library 18014, a securitylibrary 18016, and/or any other library for interpreting various typesof input data. For example, the thalamus service 18000 may use anetworking library 18014 to parse, interpret, extract, and/or otherwiseprocess network data (e.g., data received from networking sensors ordevices, networking analyses, networking reports, network database data,etc.). Similarly, the thalamus service 18000 may use a security library18016 to parse, interpret, extract, and/or otherwise process securitydata (e.g., data received from security sensors or devices, securityanalyses, security reports, security database data, etc.). Inembodiments, the intake data may also be processed using an intakelearning module 18026, which may use one or more artificial intelligencetechniques to pre-process the data, generate predictive models using thedata, predict future states of the data, and/or the like. Afterprocessing using the intake application library 18012 and/or the intakelearning module 18026, the data may be ready for management by theintake data management system 18028.

The intake data management system 18028 may process the data byprioritizing 18030, formatting 18032, suppressing 18034, using an areafocus 18036, filtering 18038, and/or combining 18040 the data. Theprioritizing 18030 may involve ranking or otherwise assigning priorities(e.g., categories, numerical priority scores, etc.) such that limitedresources may be assigned to the most important data. For example, thesuppressing 18034 and/or filtering 18038 may operate based on prioritiesin order to suppress or filter out the least important data (e.g., thedata associated with a lowest priority score) in order to avoidoverwhelming limited transmission, processing, and/or analysisresources. The formatting 18032 may involve formatting data in order toallow for easier management, which may involve compressing or otherwisedropping certain parts of data to reduce the use of transmissionresources, un-compressing data to reduce the use of decompressionresources (e.g., if bandwidth is sufficient and data is important),formatting data to emphasize or de-emphasize certain aspects, orotherwise adjust formatting. In embodiments, the formatting 18032 maydepend on the prioritizing 18030 such that more important data may beformatted in order to allow for more or better analysis, while lessimportant data may be formatted in order to reduce its usage of variousresources.

In embodiments, the suppressing 18034 may involve reducing the amount ofdata, the number of destinations to which the data is transmitted, orotherwise reducing the usage of limited resources (e.g., bandwidth,processing, analysis, etc.) of the data. In embodiments, suppressed datamay be stored (e.g., in a database) and dealt with (e.g., transmitted,processed) at a later time. In embodiments, the suppressing 18034 may bebased on various factors as described above.

In embodiments, an area focus 18036 may involve increasing the attentionpaid to certain high priority data. For example, during a securityincident, security sensor data may be sent to additional destinations,processed using additional analyses, allowed additional bandwidth andprocessing power, and/or the like. In embodiments, an area focus 18036may cause the suppression or filtering of other data that is notassociated with the area focus 18036.

In embodiments, the filtering 18038 may involve ignoring, deleting, orotherwise removing data that is not important (e.g., does not match anarea focus 18036, is low priority, etc.). In embodiments, data may beinitially suppressed (e.g., reduced or stored for later), but conditionsmay further change, causing the data to be filtered (e.g., deleted,ignored). Thus, intake data management system 18028 may allow for aprogressive downgrade of data by first suppressing and later filteringthe data depending on conditions.

In embodiments, the combining 18040 may include combining various typesof data in order to provide better analyses, generate new data, reducethe volume of data (e.g., by combining multiple data values into asingle data value), improve the quality of data (e.g., by averagingdifferent sensor readings to obtain a more accurate average reading),and/or the like. In some embodiments, lower priority data may becombined with other data in order to reduce requirements. Additionallyor alternatively, higher priority data may be combined with other datain order to improve data quality.

In embodiments, the intake data management system may interface with anintake controller 18008 and/or an intelligence system 18042. Theintelligence system 18042, for example, may use various artificialintelligence techniques to perform the intake data management (e.g.,prioritize the data, format the data, suppress the data, select an areafocus and/or assign data to an area focus, filter the data, combine thedata, etc.), predict the outcomes of intake data management, predictfuture data values, and/or the like. Additionally or alternatively, theintake controller 18008 and/or an intelligence system 18042 may operatein accordance with configured thalamus parameters 18044, which maygovern the intake data management, the artificial intelligencetechniques (e.g., the parameters may be model parameters for AI models),and/or otherwise configure the operations of the intake managementsystem 18006.

In embodiments, the control system 18002 may, in some cases, use aquantum computing service 18046, which may provide quantum computingresources to more quickly process large volumes of data, use quantummodels, and/or the like.

The control system 18002 may further comprise one or more datainterfaces 18048 for receiving data from various data sources 18004 andtransmitting the data (e.g., after intake data management) to variousdestinations. In embodiments, the control system 18002 may include othersystem subsystems 18050, such as analysis subsystems, various processingchips, or any other subsystems that may use the managed data to makedecisions, generate analyses, or otherwise perform data operations. Inembodiments, an intelligence service 18010 may operate to route themanaged data to various other system subsystems 18050, or otherwiseperform initial and/or final processing on the data.

In embodiments, the SOS control system 18002 can override the thalamusfiltering and decide to focus on a different area for any specificreason. For example, during a security incident, the SOS control systemmay route around thalamus filtering (which might normally deprioritizedata from security sensors) in order to ensure that data from securitysensors are delivered in full without any de-prioritization,suppression, filtering, etc. As another example, during regularinspections of equipment, sensor data that measures operation of theequipment (e.g., vibration sensor data) may be un-suppressed, even ifthe data appears to be within normal parameters and therefore mightusually be suppressed or filtered.

In embodiments, the control system 18002 may include a PMCP deviceinterface 18052, which may be used to transmit and/or receive data usingPMCP. Details of a PMCP device interface are further shown within asecond PMCP device interface 18060. In embodiments, the PMCP deviceinterface 18052 may be in communication with the PMCP device interface18060. The PMCP device interface 18052 may have the same components asshown within the PMCP device interface 18060.

In embodiments, the PMCP device interface may be used to convert data toa vectorized format prior to transmission. In these embodiments, avector may be considered an example of a simple predictive model (e.g.,a vector may indicate an amount of change and a direction of change fora data value, thus predicting a future state of a data value if thechange continues). For example, the conversion of a long sequence ofoftentimes similar data values into a vector indicating an amount anddirection of change, which may imply a future state of the data values,makes the communication of the data values both smaller in size andforward looking in nature.

In embodiments, PMCP may use various types of predictive models topredict current and future data values, including weighted movingaverage; Kalman filtering; exponential smoothing; autoregressive movingaverage (ARMA) (forecasts depend on past values of the variable beingforecast, and on past prediction errors); autoregressive integratedmoving average (ARIMA) (ARMA on the period-to-period change in theforecasted variable); extrapolation; linear prediction; trend estimation(predicting the variable as a linear or polynomial function of time);growth curve (e.g., statistics); and recurrent neural network basedforecasting.

Using the PMCP protocol, instead of traditional streams where individualdata items are transmitted, vectors representing how the data ischanging or what are the forecast trends in the data are communicated.The PMCP system may transmit actual model parameters to receiving unitssuch that edge devices can apply the vector-based predictive models todetermine future states. For example, each automated device in a valuechain network may be configured to train a regression model or a neuralnetwork, constantly fitting the data streams to current input data. Insome embodiments, automated devices leveraging the PMCP system are ableto react in advance of events actually happening, rather than, forexample, waiting for the depletion of inventory for an item to occur.Continuing the example, the stateless automated device can react to theforecast future state and make the necessary adjustments, such asordering more of the item.

In embodiments, the PMCP system enables communicating vectorizedinformation together with model parameters that allow predictive modelson a receiving end to predict probabilities of future values. Thevectorized information may be transmitted and processed to determine anumber of probability-based states. For example, motion vectors andmodel parameters for predicting future locations based on motion vectorsmay be transmitted using PMCP, and a receiving location may use themotion vectors as inputs to a parameterized predictive model (e.g., amodel that determines future locations of an item using the modelparameters), which may generate probabilities that an item associatedwith a motion vector is in different locations. As another example, thePMCP system may support communicating vectorized sensor readingstogether with model parameters that allow current and/or future sensorreadings to be predicted. Applied in an environment with large numbersof sensors with different accuracies and reliabilities, theprobabilistic vector-based mechanism of the PMCP system allows largenumbers, if not all, data streams to be used to produce refined modelsrepresenting the current state, past states and likely future states ofvalue chain items (e.g., goods, services, and/or the like).Approximation methods may include importance sampling, and the resultingpredictive model may be a particle filter, condensation algorithm, MonteCarlo localization, or other suitable models.

In embodiments, the vector-based communication of the PMCP system allowsdevices and/or other systems to anticipate future security events. Forexample, a set of simple edge devices may be configured to runsemi-autonomously using PMCP to generate and transmit model parametersbased on locally-sensed security data. In this example, the edge devicesmay be configured to build a set of forecast models showing trends inthe data. The parameters of this set of forecast models may betransmitted using the PMCP system. In this example, the edge devices maybe configured to build a set of forecast models showing trends in thedata. The parameters of this set of forecast models may be transmittedusing the PMCP system so that the security data may be rebuilt and usedto predict future states at a receiving device.

In embodiments, security systems may generate and transmit vectorsshowing changes in state, as unusual events tend to cause one or morevectors to show unusual patterns. In a security setting, detectingmultiple simultaneous unusual vectors may trigger escalation and aresponse by, for example, a control tower or other systems in the systemof systems. In addition, one of the major areas of communicationsecurity concern is around the protection of stored data, and in avector-based system data may not need to be stored (or may be stored onfewer devices), so the risk of data loss is removed or reduced.

In embodiments, PMCP data can be directly stored in a queryable databasewhere the actual data is reconstructed dynamically in response to aquery. In some embodiments, the PMCP data streams can be used torecreate the fine-grained data so they become part of an ExtractTransform and Load (ETL) process.

A PMCP device interface may include several modules including atransceiver module 18062, a modelling module 18064, a library module18066, and a storage module 18068. The transceiver module may include adata transceiver 18070 that may be used to transmit/receive data,including various data from data sources 18004 and/or PMCP data (e.g.,vectors, model parameters, etc.) to/from other PMCP device interfaces(e.g., PMCP device interface 18052) and/or to/from other components of asystem including the PMCP device interface. In embodiments, thetransceiver module 18062 may include an intelligence system 18072, whichmay use artificial intelligence techniques to assist in transmissionand/or reception processing. For example, the intelligence system 18072may route various types of incoming and outgoing data, prioritize ordeprioritize transmitted and/or received data from data sources 18004 vsPMCP data, and/or the like. The intelligence system 18072 may furtherinclude a PMCP controller 18074, which may understand PMCPtransmissions, parse PMCP data, and provide the received PMCP data tothe modelling module for further operations.

The modelling module 18064 may be responsible for various operations ina transmission role and/or in a receiver role. In a transmission role,the modelling module 18064 may continually receive data from variousdata sources 18004 (e.g., sensors 18022) and continually generate and/orrefine models that predict future states of the incoming data. Thevarious models may be, for example, classification models, behavioralanalysis models, prediction models, data augmentation models, and/or anyother types of model. Model parameters (e.g., neural network weights)from the generated/refined models may then be transmitted to receivers,which may use the parameters to perform classifications, behavioranalysis, prediction, augmentation and/or the like without needing tohave access to the data stream. Accordingly, in a receiver role, themodelling module 18064 may use various parameters received from anotherPMCP device interface to parameterize various types of models, then usethe parameterized models to generate data for further use by thereceiving device.

In embodiments, the PMCP device interface may train and/or executeclassification models 18076, which may be trained using data capturedfrom data sources 18004 generate various labels or classifications. Forexample, classification models may be used to output various states orconditions based on input data, including predicted future states orconditions. By transmitting classification model parameters to areceiving device using PMCP, the receiving device may also be able topredict the future states or conditions without having to receive theinput data from the data sources 18004.

In embodiments, the PMCP device interface may train and/or executebehavior analysis models 18078, which may be trained using data capturedfrom data sources 18004 to generate various behavioral analyses andfuture behavioral data. For example, behavior analysis models may beused to output current or future actions that are likely to be taken bycertain entities and/or analyses of whether the actions are withinnormal conditions or unusual. By transmitting behavioral analysis modelparameters to a receiving device using PMCP, the receiving device mayalso be able to predict the future actions and/or analyses withouthaving to receive the input data from the data sources 18004.

In embodiments, the PMCP device interface may train and/or executeprediction models 18080, which may be trained using data streamscaptured from data sources 18004 to generate current and predicted datavalues for the data streams. For example, prediction models may be usedto output current or future sensor readings based on data captured fromsensors 18022. By transmitting prediction model parameters to areceiving device using PMCP, the receiving device may also be able topredict the sensor values without having to receive the input data fromthe sensors 18022 or other data sources 18004.

In embodiments, the PMCP device interface may train and/or executeaugmentation models 18082, which may be trained using data captured fromdata sources 18004 to generate augmented data streams. For example,augmentation models may be used to generate interpolated or extrapolatedvalues from data streams that may be missing data (e.g., due to networkinterruptions), may generate predicted sensor readings for a sensor(e.g., a broken sensor) based on sensor readings from other nearbysensors, and may otherwise augment data received from data sources 18004with additional data. By transmitting augmentation model parameters to areceiving device using PMCP, the receiving device may also be able togenerate the missing data, predicted data, or other augmented datawithout having to receive the input data from the data sources 18004.

In embodiments, the PMCP device interface 18060 may use a library module18066 containing one or more modules that may be used to assist inmodelling and/or other operations. For example, a networking module18084 may contain various data about network devices, networktopologies, network digital twins, and other network data that may beleveraged to train various models, to perform ETL operations asdescribed in more detail below, or to perform other such processing. Asanother example, a security module 18086 may contain various data aboutsecurity devices, building layouts (e.g., for building securitysystems), maps, topologies, digital twins, vulnerabilities, and othersecurity data that may be leveraged to train various models, to performETL operations as described in more detail below, or to perform othersuch processing for security reasons. Various other specific modules maybe provided to enable or support specific use cases.

In embodiments, a storage module 18068 may provide various operationsfor processing data for storage and/or storing data. An ETL interface18088 may be configured to perform exchange, transform, and load (ETL)operations for storing data in a PMCP database 18090. The PMCP database18090 may be used to store various data, including data received fromdata sources 18004 (e.g., such that historical data may be used togenerate/refine various models), as well as the models themselves, modelparameters, and/or the like.

In embodiments, the thalamus service and PMCP may provide complementarytechniques for managing large amounts of data. For example, PMCP mayreduce the bandwidth and storage requirements for working with largeamounts of data because PMCP may only require transmitting modelparameters, instead of transmitting bandwidth-intensive data streams.However, when dealing with large numbers of data sensors or other datasources, PMCP may not be enough to reduce data to manageable levels, asthe number of PMCP streams, number of models, etc. may still be toolarge to handle. In these cases, the thalamus service may operate toprioritize, format, suppress, filter, or combine PMCP data streams inorder to allow for a focus on the most important PMCP data streams atany given time. Several benefits are realized by combining thetechniques in this manner. For example, although massive amounts of datamay be collected, PMCP may allow the communication of model parametersfor predicting some or all of the data, and the thalamus service mayallow for a focus on the most important models and predictions at anygiven time. Moreover, the use of PMCP causes the data to be inherentlypredictive and thus forward-looking, which, in combination with thethalamus service, allows for a focus on the most important data beforethe occurrence of potential issues that may need various actions (e.g.,interventions, maintenance, purchase orders, supply adjustments,estimate adjustments, etc.).

FIG. 163 shows the interaction of the intake controller 18008, intakemanagement system 18006, and various other components of the thalamusservice 18000 with PMCP according to some embodiments of the invention.In the illustrated embodiments, inputs may be received to the intakecontroller 18008 from different sources. For example, a first source ofdata may include various sensors, external systems, process data, andother such data 18102 that may be received from various data generators,data analysis systems, and other data outputs outside of the thalamusservice. Additionally or alternatively, a second source of data mayinclude one or more preconfigured PMCP devices with location processing,which may provide at 18104 that may include PMCP model parameters,vectorized data, or other PMCP data.

The intake controller 18008 may ingest the data and determine whetherthe data is PMCP data or not at a decision 18106. If the data is notPMCP data, then the intake controller 18008 may determine if the datahas been reduced or not. If the data has not been reduced, then the datamay be sent to the intake management system for processing (e.g.,prioritization, formatting, suppressing, area focus, filtering,combining, etc. as discussed above). In other words, if the data has notalready been reduced in some way (e.g., either via PMCP or using otherdata reduction techniques), the data may be processing and potentiallyfiltered, suppressed, or otherwise reduced. Thus, the thalamus servicemay provide one data reduction techniques that may be used in additionto or as an alternative to other data reduction techniques, which mayinclude PMCP.

If the data was not PMCP data but was reduced as determined at 18108, orif the data was PMCP data as determined at 18106, then the intakecontroller 18110 may determine whether the thalamus service is acting asa PMCP consumer for the data. If so, the data may be sent to the PMCPdevice interface 18052 for reception and processing (e.g., modelling,prediction, etc.). If not, then one or more ETL processes may be used at18114 to extract, transform, and load the data into the PMCP database.

Whether the data is processed by the PMCP device interface 18052 orusing ETL processes at 18114, the resulting data may then be provided todownstream system of systems data consumers for further processing at18116.

PMCP and thalamus service techniques may be used (together orseparately) in a wide variety of embodiments. In embodiments where edgedevices are configured with very limited capacities, additional edgecommunication devices can be added to convert the data into PMCP format.For example, to protect distributed medical equipment from hackingattempts, many manufacturers will choose to not connect the device toany kind of network. To overcome this limitation, the medical equipmentmay be monitored using sensors, such as cameras, sound monitors, voltagedetectors for power usage, chemical sniffers, and the like. Functionalunit learning and other data techniques may be used to determine theactual usage of the medical equipment detached from the networkfunctional unit, generate vectorized data therefrom, and/or transmitvarious model parameters using PMCP. On the receiving end, a thalamusservice may receive the vectorized data and/or model parameters, may usethalamus techniques to determine whether the PMCP data and/or other datareceived from other medical devices should be prioritized, filtered,suppressed, or the like, may predict future states of the medicalequipment based on the PMCP data, and may use any or all of the data totake various actions, perform various analyses, and the like.

In some embodiments, communication within the value chain usingvectorized data allows for a value chain to have a constant view of whatthe likely future state is. These techniques allow for future states tobe communicated to the value chain, thus allowing value chain entitiesto respond ahead of future state requirements without needing access tofine-grained data.

In some embodiments, the PMCP protocol can be used to transmit andreceive relevant information (e.g., important or high priorityinformation, as determined by a thalamus service) about productionlevels and future trends in production to various external entities. Insome of these example embodiments, a PMCP data feed may be used for dataobfuscation (e.g., communicating sensitive data as vectorized dataand/or model parameters). For example, PMCP allows real contextualinformation about production levels to be shared with consumers,regulators, and other entities external to the value chain networkwithout the direct sharing of sensitive data values. For example, when acustomer chooses to purchase a new car, one or more value chain entitiesmay be integrated into the selection process and may determine (e.g.,based on predictive models) that there is an upcoming shortage of redpaint. In this case, the value chain entities could communicate PMCPdata that would be processed and used to show the customer device theimpact of different choices on delivery time, without providingsensitive data to the customer device or other external entity.

PMCP and vectorized data processes further enable simple data-informedinteractive systems that a user can apply without having to buildenormously complex big data engines. As an example, an upstreammanufacturer may have an enormously complex task of coordinating manydownstream consumption points. Through the use of PMCP and/or thalamusservices, the manufacturer may be able to provide real information toconsumers without the need to store detailed data and build complexmodels, which may require setting up large-scale systems for processinglarge amounts of data and the like.

In embodiments, edge device units may communicate via the PMCP system toshow direction of movement and likely future positions. For example, amoving robot can communicate its likely track of future movement. Inembodiments involving large numbers of moving robots, a thalamus servicemay determine which robots need to be prioritized and monitored closely(e.g., because they are moving outside of prescribed boundaries,behaving in unpredictable ways, etc.).

In embodiments, the PMCP system and/or thalamus system enables visualrepresentations of vector-based data (e.g., via a user interface),including highlighting of areas of concern without the need to processenormous volumes of data. The visual representation allows for thedisplay of many monitored vector inputs. The user interface can thendisplay information relating to the key items of interest, specificallyvectors showing areas of unusual or troublesome movement. This mechanismallows sophisticated models that are built at the edge device edge nodesto feed into end user communications in a visually informative way.

As can be appreciated, functional units produce a constant stream of“boring” data (e.g., data that does not change, changes slightly, orchanges very predictably). By changing from producing data, tomonitoring for problems, issues with the logistical modules arehighlighted without the need for scrutiny of fine-grained data. Inembodiments, PMCP device interfaces may constantly generate and/orrefine a predictive model that predicts a future state. In the contextof maintenance, refinements to the parameters in the predictive modelare in and of themselves predictors of change in operational parameters,potentially indicating the need for maintenance. Moreover, thecommunication of operational parameters for large numbers of devices maybe processed by a thalamus service such that data for devicesfunctioning normally may be filtered or suppressed until conditionschange.

In embodiments, functional areas are not always designed to be connectedto the value chain network, but by allowing for an external device tovirtually monitor devices, functional areas that do not allow forconnectivity can become part of the information flow in the value chaingoods. This concept extends to allowing functional areas that havelimited connectivity to be monitored effectively by embellishing theirdata streams with vectorized monitored information. Placing an automateddevice in the proximity of the functional unit that has limited or noconnectivity allows capture of information from the devices without therequirement of connectivity. There is also potential to add trainingdata capture functional units for these unconnected or limitedlyconnected functional areas. These training data capture functional unitsare typically quite expensive and can provide high quality monitoringdata, which is used as an input into the proximity edge devicemonitoring device to provide data for supervised learning algorithms.

Oftentimes, value chain network locations are laden with electricalinterference, causing fundamental challenges with communications. Thetraditional approach of streaming all the fine-grained data is dependenton the completeness of the data stream. For example, if an edge devicewas to go offline for 10 minutes, the streaming data and its informationwould be lost. With vectorized communication, the offline unit maycontinue to refine the predictive model until the moment when itreconnects, which allows the updated model to be transmitted via thePMCP system.

In embodiments, value chain network systems and devices may be based onthe PMCP protocol. For example, value chain network cameras and visionsystems (e.g., liquid lens systems), user devices, sensors, robots,smart containers, and the like may use PMCP and/or vector-basedcommunication. By using vector-based cameras, for example, onlyinformation relating to the movement of items is transmitted. Thisreduces the data volume and by its nature filters information aboutstatic items, showing only the changes in the images and focusing thedata communication on elements of change. The overall shift incommunication to communication of change is similar to how the humanprocess of sight functions, where stationary items are not evencommunicated to the higher levels of the brain.

Radio Frequency Identification allows for massive volumes of mobile tags(e.g., cargo RFID tags for cargo being transported by a smart container)in a value chain network to be tracked in real-time. In embodiments, themovement of the tags may be communicated as vector information via thePMCP protocol, as this form of communication is naturally suited tohanding information regarding the location of tag within the value chaingoods. Adding the ability to show future state of the location usingpredictive models that can use paths of prior movement allows the valuechain goods to change the fundamental communication mechanism to onewhere units consuming data streams are consuming information about thelikely future state of the value chain goods. In embodiments, eachtagged item may be represented as a probability-based location matrixshowing the likely probability of the tagged item being at a position inspace. The communication of movement shows the transformation of thelocation probability matrix to a new set of probabilities. Thisprobabilistic locational overview provides for constant modeling ofareas of likely intersection of moving units and allows for refinementof the probabilistic view of the location of items within the valuechain network. Moving to a vector-based probability matrix allows unitsto constantly handle the inherent uncertainty in the measurement of thestatus of value chain network items, entities, and the like. Inembodiments, status includes, but is not limited to, location,temperature, movement and power consumption.

In embodiments, continuous connectivity is not required for continuousmonitoring of sensor inputs in a PMCP-based communication system. Forexample, a mobile robotic device with a plurality of sensors cancontinue to build models and predictions of data streams whiledisconnected from the network, and upon reconnection, the updated modelsare communicated. Furthermore, other systems or devices that use inputfrom the monitored system or device can apply the best known, typicallylast communicated, vector predictions to continue to maintain aprobabilistic understanding of the states of the value chain goods.

Energy Systems and Processes

The disclosure relates to energy systems and processes. In exampleembodiments, there is an energy system and process (e.g., also referredto as energy system which may be or may include an energy system,process, module, service, platform, and/or the like). In exampleembodiments, as shown in FIG. 164 , there is an energy system 19000 thatmay interact with any system, subsystem, component, process, platform(e.g., the value chain network management platform), and the like asdescribed in the disclosure. In some examples, the energy system 19000may be a separate system that is external to systems, subsystems,components, processes, platforms, etc. in the disclosure. In otherexamples, the energy system 19000 may be integrated with any one of thesystems, subsystems, components, processes, platforms, etc. in thedisclosure. For example, as shown in FIG. 164 , the energy system 19000communicates with the value chain network entity 652 and the datahandling layers 608 (e.g., including the value chain management platform604, adaptive intelligent systems 614, monitoring systems 808, datastorage systems 624, interfaces 702, connectivity facilities 642, and“process and application outputs and outcomes” 1040). The energy system19000 (e.g., including at least one energy process) may utilize anenergy model. The energy system 19000 may be part of a group of valuechain building blocks that may be combined with various other processesand/or systems within the enterprise control tower. In exampleembodiments, the energy system 19000 may provide modular adaptiveresource package technology (e.g., having energy, energy computing,and/or energy networking processes). For example, the energy system19000 may relate to energy storage on a modular level across a networkof energy storage systems and devices (e.g., use of modular energystorage). The energy system 19000 may address various needs for powermanagement across communities, businesses/companies, organizations,colleges/universities, etc. This may be accomplished, for example, bymodularization of power storage. In example embodiments, where there maybe limited power resources and a need to focus on renewable energy,optimization and modularization of power storage may address theseissues.

In example embodiments, the energy system 19000 may include and/orutilize any one or more of the following technologies, systems, and/orprocesses: 3-dimensional (3d) printing of batteries, a battery energystorage system (BESS), various battery types, coordination processes,decentralized energy grids, energy pricing, energy storage technology,energy-as-a-service such as an energy-as-a-service system (e.g., energydistributed and localized), energy-related sectors and transactions,machine learning (ML) and/or artificial intelligence (AI) for energyoptimization, ML/AI for automation, ML/AI for matching energyutilization/demand to energy production across a distributed network(e.g., network of energy production, storage, and delivery systems),quantum, renewable energy (e.g., renewable energy kit), technologies forslicing (e.g., systems and/or processes for slicing production, storage,and delivery), and the like.

In example embodiments, the energy system 19000 may include energystorage technology. In some examples, the energy storage technology mayinclude one or more types of batteries. For example, the batteries mayinclude lithium-ion batteries, flexible batteries, structural batteries,solid-state batteries (e.g., technology advancements may lead toemergence or re-emergence in some cases of solid-state batteries ascommercial alternatives), and/or flow batteries.

In example embodiments, new materials and manufacturing methods may haveresulted in the introduction of flexible batteries (e.g., flexibleprimary and secondary cells), and a pipeline of new possibilities. Usecase or product driven conformable and conforming capabilities may offera new variable to optimize product design. Flexible batteries may bedesigned as an integral part of a product rather than as an add-onmodule, and may be adapted for clothing and other wearable electronics,medical devices, drug delivery systems, micro IoT devices, flexibleelectronic devices that incorporate both flexible circuits andbatteries, etc.

In example embodiments, structural batteries may use carbon fiber as anegative electrode and lithium iron phosphate-coated aluminum foil as apositive electrode. Tradeoffs between battery weight and productperformance may sometimes eliminate or limit battery powered productcategories. Technological advances may provide opportunities for designsthat incorporate at least one structural battery into a product itself.Batteries that employ carbon nanotube electrodes as structural elementsmay provide design flexibility and opportunities for an overall weightreduction, for example, as part of a hull of a vessel. Structuralstorage elements may support integrated systems for battery-powered, andpossibly grid-independent infrastructure, vehicles, devices, etc., thatmay incorporate transactions capabilities. Example uses may includesidewalks, roads, airports, etc. (public or private). Shape, cost,structural requirements, design life, thermal management, power andenergy, or a combination of these features may be automated as part of adesign and value chain consideration process.

In example embodiments, flow batteries may be used to decouple energyand power. For example, using a type of flow batteries that decoupleenergy and power may offer some unique design and integrationopportunities such as purpose-built buildings and infrastructure. Theseflow battery systems may provide a near-term alternative to lithium-ionbatteries. New chemistries such as organic formulations may lead toeasier use of abundant and less corrosive electrolytes that may makethis technology less costly.

In example embodiments, the energy storage technology may include smartbatteries. The smart batteries may be smart batteries with a batterymanagement system (BMS) and other functions down to a cell-level. TheBMS at the cell-level may be used to manage charge, discharge, voltagebalancing, and the like. In other examples, the smart batteries may besmart batteries with cell-level monitoring and data streams. In otherexamples, the smart batteries may be smart batteries with cell-leveldistributed energy management. In other examples, the smart batteriesmay be smart batteries with energy management on a chip (e.g., chipset)for cell-level or system level control. In example embodiments,automated battery assembly monitoring, maintenance, and performancemanagement may be simpler when cells monitor themselves, freeingassembly control to perform higher level operations. For example, pullBMS and other functions down to the cell-level may manage charge,discharge, voltage balancing, etc. In example embodiments, smartbatteries may provide cell-level monitoring and data streams, cell-leveldistributed energy management, and/or energy management on a chip forcell or system level. In example embodiments, energy management on achip may provide cell or system level control. For example, a chip maybe used that electrically switches cell connections to optimize abalance of power and energy requirements. Individual cells may requesttheir own replacement or take themselves off-line. In exampleembodiments, smart batteries may utilize quantum computing for design orreal-time operating temperature optimization, including automated designand system control. Smart batteries may also provide vibration controlsuch as controlled vibration to manage dendrites and improve batterylife for lithium-ion, zinc, and others.

In example embodiments, the energy storage technology may includevarious controls and/or management functions. For example, the energystorage technology may provide controlled vibration to manage dendritesand improve battery life (e.g., for lithium ion, zinc, and others). Theenergy storage technology may provide battery product lifecyclemanagement (e.g., a battery product lifecycle management system) and/orbattery management and control (e.g., battery having wireless power andcontrol).

In example embodiments, battery product lifecycle management may addressconcerns of supply chain optimization for primary and intermediatebattery materials and related opportunities. Battery product lifecyclemanagement may be tied to more vertically integrated operations,including consolidation of processes, co-location of battery productionand products, and the like. In example embodiments, battery productlifecycle management may be used with battery manufacturing thatincludes material use optimized with value chain network (VCN) modellingand/or battery manufacturing with 3D printed materials and processes. Inexample embodiments, battery product lifecycle management may includedata collection, management, and analysis that may incorporate testingand tracking of battery “cell” and other sub-components for VCNoptimization. Battery product lifecycle management may also be used withbattery disposal, carbon footprint management, etc. as well as withbattery recycling and reuse (e.g., lithium and/or cobalt recycledmaterials).

In example embodiments, battery management and control such as wirelesspower and control may include wireless technology for all levels ofbattery implementation and control. This may provide design andoperational flexibility, for example, charge/discharge control,real-time power/energy configurations, and/or operational notificationsdown to a cell-level. In example embodiments, wireless power and controlmay include BMS software to support simplified system integration,system and software standardization, integrated higher level powerdispatch and control systems, and/or chip-level integrated circuits andpower management.

In other examples, the energy storage technology may utilize abattery-powered/grid-independent infrastructure. In example embodiments,structural storage elements may support integrated systems forbattery-powered, and possibly grid-independent infrastructure, vehicles,devices, etc., that may incorporate transactions capabilities. Examplesmay include sidewalks, roads, airports, etc. (public or privateinfrastructures). In example embodiments, shape, cost, structuralrequirements, design life, thermal management, power and energy, or acombination of these may be automated as part of a design and valuechain consideration process.

In example embodiments, the energy storage technology may utilizehigh-performance electrodes and/or high-performance separators. Inexample embodiments, high-performance electrodes may include grapheneand/or nanotubes. These high-performance electrodes may allow for fastercharging and discharging, fewer thermal management issues and associatedsafety, cycle life, and other performance improvements. Electrodeadvancements may benefit nearly all battery types, and they mayrepresent design opportunities for more and higher performing productimplementations. Several advances highlight electrode improvements suchas examples including various carbon configurations like graphene andnanotubes, and other materials that may increase active surface area,provide better manufacturability, provide lower resistance, longer life,etc. In example embodiments, high-performance separators may be usedwith some example battery technologies. For example, with some batterytechnologies, especially flow batteries and fuel cells, the separatormay be a key element that allows charge transfer without direct mixingof anolyte and catholyte constituents. Improved separators may result inmore efficient battery operation, lower costs, and wider deployment.

In example embodiments, the energy storage technology may utilizeorganic flow battery electrolytes and/or polymer lithium-ionchemistries. For example, the energy storage technology may include abattery having organic flow battery electrolytes. There may be a seriesof incremental improvements relating to organic flow batteryelectrolytes that may be rolled into existing and improvedinfrastructures. Current research may focus on low-cost andenvironmentally friendly options. Examples may include organic flowbattery electrolytes, polymer lithium-ion chemistries, and the like. Inexample embodiments, the energy storage technology may include a batterywith polymer lithium-ion chemistries.

In example embodiments, the energy storage technology may utilize waveenergy (e.g., system for storing wave energy) and/or thermal energy(e.g., system for storing thermal energy). For example, ocean andgeothermal open and closed systems may provide interesting deployments.In example embodiments, the energy storage technology may providegravity energy storage (e.g., system for storing gravity energy). Forexample, gravity energy storage technology may be integrated intobuilding and infrastructure projects and may be managed as part of anintegrated energy management system. In example embodiments, the energystorage technology may provide carbon particles that create current byinteracting with surrounding organic solvent (e.g., system forgenerating energy having carbon particles that may create current byinteracting with a surrounding organic solvent).

In example embodiments, the energy system 19000 may include variousbattery types. These battery types may include a zinc battery type, anickel battery type, and/or a cobalt battery type.

In example embodiments, the energy system 19000 may include systemsand/or processes for providing battery energy storage. This may relateto a battery energy storage system (BESS). Some example BESStechnologies with existing deployments and other near-term possibilitiesmay be hydrogen fuel cells and various types of flow batteries (e.g.,vanadium-based batteries). Vanadium-based batteries may be vanadiumredox batteries (VRBs) (e.g., also known as vanadium flow batteries(VFBs) or vanadium redox flow batteries (VRFBs)) which are a type ofrechargeable flow battery. Other examples of BESS technologies mayinclude pumped hydro, gravity, thermal, tidal, and waves.

In example embodiments, the BESS may be integrated with a buildingenergy management system. In the US, there is a “Standard for theInstallation of Stationary Energy Storage Systems” that has providedcover for lithium-ion and other energy storage deployments in higherdensity population zones and commercial buildings with shared tenants(e.g., New York City). There may be some drawbacks to using lithium-iontechnologies such as safety, mining of raw materials, recycling, cyclelife, total cost of ownership, quality control, temperature-drivenperformance limitations, energy density, etc. The standard, drawbacks,and other factors may be spurring a range of industry convergences suchas BESS integration with building energy management (BEM) systems.

In some example embodiments, the BESS may be a flow battery-based BESS.Flow batteries may be the next nearest commercial-scale large-scaleBESS. Flow battery systems may use most of the same power, control, anddata infrastructures used with lithium-ion deployments. Flow batterysystems may not pose a fire hazard, have fewer limitations oncharge/discharge cycles, and may have a lower cost of ownership over a20-to-30-year span compared to lithium-ion systems. A typical flowbattery electrolyte may be easily recycled or reused, and in somefinancing models, the flow battery electrolyte may be leased to reduce acost of implementation. Technical advances associated with lithium-ionbatteries such as power electronics, controls, electrodes, separators,and in some cases electrolyte, may also provide performance and costimprovements for flow battery systems.

In example embodiments, the energy system 19000 may include systemsand/or processes for providing 3d printing of batteries. This mayutilize a 3d printer for printing batteries resulting in 3d printedbatteries of various types. Supply chain optimization for primary andintermediate battery materials may be a concern and opportunity that maybe tied to more vertically integrated operations that may includeconsolidation of processes, including co-location of battery productionand products. In example embodiments, battery manufacturing may includematerial use optimized with value chain network (VCN) modelling. Inother examples, battery manufacturing may include 3d printed materialsand processes. In example embodiments, data collection, management, andanalysis may incorporate testing and tracking of a battery “cell” andother sub-components for VCN optimization. There may be systems and/orprocesses involving battery disposal, carbon footprint management, etc.There may be other systems and/or processes that may provide batteryrecycling and reuse (e.g., lithium-ion battery recycling and reuse suchas recycling and reuse of lithium).

In example embodiments, the energy system 19000 may include therenewable energy technology (e.g., renewable energy kit). This mayrelate to a system for generating, storing, and/or using renewableenergy (e.g., renewable energy kit/in-a-box).

In example embodiments, energy provider(s) may include a variety ofoptions such as purchasers, servicers, self-generated, private/public,and/or a mixed combination. In example embodiments, energy source(s) mayinclude a variety of options such as solar, wind, batteries, thermal,gravity, waves, and/or a grid.

In example embodiments, the energy system 19000 may includedecentralized energy grids. These decentralized energy grids may includesafety systems for decentralized virtual grids. In some examples, thedecentralized energy grids may include control systems for decentralizedvirtual grids.

In other examples, the decentralized energy grids may allow fortransactions between end users. In example embodiments, a decentralizedenergy grid may allow for different sources of energy generation (e.g.,solar, hydro, wind) and may allow for transactions between end users forexcess energy that is produced and not needed. The energy assets of ahousehold may be tokenized and bought or sold on a decentralizedmarketplace. The transactions may be between users belonging to the samegeographic location or different locations and may consider pricearbitrage. The excess energy may be supplied to the energy grid throughsmart contracts. The energy data (e.g., personal or aggregate data in acluster) may be monetized by selling usage data.

In example embodiments, the energy system 19000 may include systemsand/or processes for providing energy-related sectors and transactions(e.g., energy transactions). “Energy-related sectors and transactions”may incorporate and be used to refer to these systems and/or processesfor providing energy-related sectors and transactions throughout thedisclosure. For example, the energy system 19000 may include an energytransaction system for facilitating energy transactions between parties.The energy-related sectors and transactions may provide local andregional energy arbitrage (e.g., using a local and/or regional energyarbitrage system). The energy-related sectors and transactions may alsoprovide local and regional energy management (e.g., using a local and/orregional energy management system). In some examples, the energy-relatedsectors and transactions may provide an energy data marketplace (e.g.,using an energy data marketplace for personal or aggregated monetizationof energy data). In other examples, the energy-related sectors andtransactions may include kiosks and/or microservices for energy inremote or underserved areas. In example embodiments, the energy-relatedsectors and transactions may include a private carbon usage monitoringand management system (e.g., system for monitoring and management ofprivate carbon usage). In other examples, the energy-related sectors andtransactions may include an enterprise carbon usage monitoring andmanagement system (e.g., system for monitoring and management ofenterprise carbon usage and/or energy carbon usage). The energy-relatedsectors and transactions may include a solar powered pump and/or batterysystem for crop irrigation that may support smart contracts. In someexamples, the energy-related sectors and transactions may provideautomated financing/payments/insurance mechanisms and/or smart contractsthat may support private energy infrastructure investments (e.g., asystem for automating financing, payments, insurance mechanisms, and/orsmart contracts that support private energy infrastructure investments).In example embodiments, the energy-related sectors and transactions mayinclude a gaming engine smart contract energy management platform. Thegaming engine smart contract energy management platform may beconfigured to enable energy management, energy visualization, modelingenergy options, and/or smart contract execution.

In example embodiments, the energy-related sectors and transactions mayinclude energy transactions between parties which may relate to energyownership concepts and markets outside a utility penumbra, local andregional energy arbitrage, and/or local and regional energy management.In other examples, the energy-related sectors and transactions mayinclude monetization of data such as personal or aggregated data. Inexample embodiments, the energy-related sectors and transactions may beutilized in remote or underserved areas (e.g., usingkiosks/microservices). In other example embodiments, the energy-relatedsectors and transactions may provide private and commercial lifecyclecarbon monitoring and management. This may provide optimization ofenergy use mix, real-time cost offsets based on time of day, incentives,regional regulations, etc., and/or may be part of a personal valuechain. In example embodiments, the energy-related sectors andtransactions may include integrated purpose-built systems (e.g., a solarpowered pump/battery system for crop irrigation that supports smartcontracts). In other example embodiments, the energy-related sectors andtransactions may provide automated financing, payments, insurancemechanisms, and associated smart contracts that support private energyinfrastructure investments. The energy-related sectors and transactionsmay also include a gaming engine smart contract energy managementplatform. Combined technologies in distributed hubs such as datacenters, communications, power generation, storage, and dispatch maycreate multiple complex optimization scenarios. Gaming engines may beused for energy management, visualization, and contract execution. Thismay be a platform that is embedded or stand-alone, and may be licensed,or subscription based. Applications of the gaming engines may includebut may not be limited to: multi-tenant buildings, residential energypurchasing, residential use decisions, residential visualization, gamingengines embedded in products, and/or model energy options using gamingengines with smart contracts.

The energy-related sectors and transactions may provide energymanagement (e.g., using an energy management system). In exampleembodiments, the energy-related sectors and transactions may provideintegration of multiple energy sources for storage and dispatch (e.g.,using the energy management system). The energy-related sectors andtransactions may also provide a deployable integrated and modular energystorage system that incorporates interchangeability. For example, theenergy management system may include an integrated and modular energystorage system that incorporates interchangeability.

In example embodiments, the energy management system may provide energymanagement where new technology, lower cost, advancing regulations,blockchain distributed ledger, electric vehicle integration, buildingenergy management applications, etc. may encourage wider and acceleratedadoption. The energy management system may provide integration ofmultiple energy sources for storage and dispatch (e.g., co-located orotherwise). Also, the energy management system may provide deployableintegrated and modular energy storage systems that may incorporateinterchangeability (e.g., family of parts similar to parts associatedwith power tools). The energy management system may include and/or beutilized with commercial and building management systems. With theadoption of electric vehicles and large-scale BESS, the energymanagement system may provide smaller scale distributed and islandedstorage/plus opportunities. Many smaller-scale islanded/grid-connectedsystems may be used to address costs of transporting fuel oil to andmaintaining diesel generators at remote sites such as islands, miningsites, etc. The energy management system may provide packaged systems orportions of packaged systems for a wider set of customers that mayinclude integration of various generating assets (e.g., wind, solar,diesel), storage, monitoring, and control. In example embodiments, theenergy management system may include and/or be incorporated withresidential home/community systems. For example, integrated control,energy management, transaction, and market enabling technologies forcommercial and residential multi-tenant installations may includestationary batteries as well as electric vehicles and their batteries,which may become more sophisticated. In example embodiments, the energymanagement system may include new batteries for residential andcommercial storage (e.g., using smart batteries). In exampleembodiments, the energy management system may provide energy servicescontracting. For example, energy services contracting may relate to anindependent service industry associated with installation and operationof residential solar systems that may expand to include variousintegrated energy storage and service options. There may beopportunities for a wide range of contracted services.

In example embodiments, the energy-related sectors and transactions mayinclude a platform for dynamic allocation of distributed data centerresources. For example, the platform may include a system for allocatingresources based on energy cost, environmental impact, transactionvolume, transaction type, and/or transaction priority. Theenergy-related sectors and transactions may also include an integratededge-based system that may generate and/or store energy. In exampleembodiments, distributed and agile data centers may be focused onworkload placement which may lead to new infrastructure changes andstrategies such as the integration of on-premises, co-location, cloud,and edge delivery options. In example embodiments, a platform fordynamic allocation of distributed data center resources may allocateresources based on energy cost, environmental impact (e.g., legislationshows movement in this area), transaction volume, type and priority oftransactions, etc. In example embodiments, integrated edge-based systemsmay be systems that generate energy, store energy, and/or providedatacenter-like services along with other energy managementcapabilities. These other energy management capabilities may includeand/or relate to: home systems, systems associated with one or moreproducts with integrating intelligence, systems that may be availablefor a subscription fee from a service aggregator, integrated 5Gcommunications infrastructure planning, and/or home-based cryptocurrencyoperations.

In some example embodiments, the energy-related sectors and transactionsmay provide analysis of land use costs. For example, the analysis ofland use costs may be accomplished by a system for analyzing costs ofrenewable deployments where the costs may be based, at least in part, onenvironmental, regulatory, and/or zoning factors. Land use may be partof an energy value chain, where environmental, regulatory, zoning, andother local concerns may become costs for renewable deployments. Thisanalysis may be used with various examples such as floating wind andsolar projects.

The energy-related sectors and transactions may also provide personalenergy management. In example embodiments, the personal energymanagement may be provided by a personal energy management system formanaging personal energy usage, storage, and/or generation. Inputs maybe personal energy assets and descriptions, personal preferences (e.g.,carbon footprint, budget), real-time data (e.g., time of day pricing,cloud forecasts, wind forecasts), infrastructure (e.g., regional rules,interconnection), asset pricing models (e.g., depreciation, operatingcosts). This may provide time segmented automated personal microgridcontrol (e.g., actively managed, set, and forget).

In example embodiments, the energy system 19000 may include coordinationfeatures. For example, these coordination features may include a systemfor coordinating energy demand across multiple distributed energyproduction, storage, and/or delivery systems. In example embodiments,the coordination features may include coordination of energy demandacross multiple distributed and partially isolated energy production,storage, and/or delivery systems.

In example embodiments, the energy system 19000 may include systemsand/or processes for providing energy pricing. Energy pricing mayinclude pricing mechanisms that incorporate security, reliability,type-slicing, and/or time-slicing into a pricing matrix. These pricingmechanisms may be used with an energy value chain network such as adecentralized network of production, storage, and delivery systemsinstead of a centralized grid.

In example embodiments, the energy system 19000 may include ML/AI forautomation which may relate to automation of energy transactions and/orenergy management. In some examples, the ML/AI for automation mayinclude an ML and/or AI system for smart contract tracking (e.g., smartcontract management) and/or pricing energy production on a blockchainsystem.

In other examples, the ML/AI for automation may include an ML and/or AIsystem configured for automation of energy management in a supply chain(e.g., automate renewable energy management in a supply chain). Forexample, renewable energy use in supply chain (e.g., factories ordistribution centers) may involve AI capabilities in tracking supply anduse of energy such that there may be a sufficient quantity of energystored for use each day. This ML and/or AI system may monitor needs overthe course of any period of time (e.g., hours, days, weeks, months,year). Fluctuations in needs may be anticipated by the ML and/or AIsystem such that if the needs reach a threshold where new additionalsources are needed, the ML and/or AI system may anticipate these needsbefore it becomes a problem for the supply chain/value chain. Renewableenergy may vary depending on location and region which may includesunlight, wind, water, geothermal heat, etc. The ML and/or AI system mayalso be utilized to make energy usage more efficient. Similar totracking needs, usage may be similarly tracked with respect to energyusage to determine where inefficiencies may be realized, and then the MLand/or AI system may make suggestions for adjustments of energy useacross supply chain/value chain with respect to the network.

In example embodiments, the energy system 19000 may include ML and/or AIfor energy optimization. The ML and/or AI for energy optimization mayfurther include and/or utilize the following systems and/or processes:an ML and/or AI system configured to optimize safety of lithium-ionbatteries, an ML and/or AI system configured to optimize cost oflithium-ion batteries, an ML and/or AI system configured to optimizerecycling characteristics of lithium-ion batteries, and/or an ML and/orAI system configured for optimizing food and energy production andstorage.

In example embodiments, the ML and/or AI system for optimizing food andenergy production may include a distributed food production value chainnetwork. This may relate to foods (e.g., whether heavy and thusrelatively expensive to transport based on energy needs) and plants(e.g., growing plants which may require relatively high energy density).Food supply chains may be highly optimized and effective in the advancedindustrialized world but may be vulnerable to disruption, and for manyfoods (such as produce and meats), a huge amount of energy may beconsumed in transporting items that may be composed largely of water.Food production may be energy-intensive and in some examples requirespecific types of energy (e.g., a given set of spectral characteristicsto promote plant growth over time). In some examples, a localized foodsupply chain may be simultaneously managed, a location provisioned, andutilization of small-scale energy production entities, energy storageentities, delivery systems, and food production systems may provide arobust, efficient, food-energy value chain network. In exampleembodiments, a platform may incorporate robotic process automation toprovision and deliver energy of a desired mix (e.g., for environmentalobjectives) within a target budget (e.g., by time-shifting) to producedesired outcomes (e.g., plant growth to meet forecast demand). This maybe achieved by DPANN techniques, quantum computing, and/or otheroptimization techniques in the disclosure. Demand-side forecasting maybe applied to consumer demand for food and for energy (e.g., based on awide variety of IoT and crowd-sourced data). This may includeaggregation of demand by robotic agents to a point that may justifyprovisioning of a production entity (e.g., mix of infrastructure andfood). Some goods may be produced with minimal footprints (such as invertical farms), while others may require more land, but in either caseland use may be optimized by the system as well, such as schedulingtemporary locations for production, storage and delivery systems forenergy and for food, and taking into account parameters of each (e.g.,energy requirements for food storage parameters).

In example embodiments, the ML and/or AI for energy optimization mayinclude an ML and/or AI system for optimizing energy utilization (e.g.,for a specific location, a time window, and an application). Forexample, source production of large quantities may be timed to meet at aparticular use window (e.g., shortly before asphalt is used), proximalto a point of use (e.g., long haul may not be a cost-effective option),and on-site application may require further energy to keep materialpliable, deposit the material, and configure the material to prepare foruse. On-site application may also require several energy consumingdevices (e.g., spreaders, rollers, compressors) and materials (e.g.,edge sealers, topcoat sealers, line striping). In example embodiments,decentralized energy systems may be configured proximal to target areasfor asphalt use (e.g., a new development, roadways such as for a town ora stretch of highway) that may need resurfacing. With flexible access toenergy and storage, a construction crew may bring required (or excessdemanded) energy with them as well.

In example embodiments, the ML and/or AI for energy optimization mayinclude and/or utilize the following systems and/or processes: an MLand/or AI system configured for optimization of power grids, an MLand/or AI system configured for design optimization (e.g., configuredfor design optimization of a battery), and/or an ML and/or AI systemconfigured for real-time operating temperature optimization (e.g.,configured for real-time operating temperature optimization of abattery). In example embodiments, quantum computing for design orreal-time operating temperature optimization may include automateddesign and system configured control.

In example embodiments, the ML and/or AI for energy optimization mayinclude and/or utilize the following systems and/or processes: an MLand/or AI system for optimizing battery disposal and/or an ML and/or AIsystem configured for optimizing battery recycling or reuse. In exampleembodiments, these ML and/or AI systems may address concerns surroundingsupply chain optimization for primary and intermediate battery materialswhich may be tied to more vertically integrated operations, includingconsolidation of processes and co-location of battery production andproducts. For example, these ML and/or AI systems may address needs forbattery manufacturing that may include material use optimized with VCNmodelling. These ML and/or AI systems may be utilized for batterymanufacturing with 3d printed materials and processes. These ML and/orAI systems may also be utilized with data collection, management, andanalysis that may incorporate testing and tracking of battery “cell” andother sub-components for VCN optimization. The ML and/or AI system foroptimizing battery disposal may provide battery disposal, carbonfootprint management, etc. The ML and/or AI system for optimizingbattery recycling or reuse may provide for battery recycling and reusesuch as with supply of lithium or cobalt needed for lithium-ionbatteries.

In example embodiments, the ML and/or AI for energy optimization mayinclude an ML and/or AI system for optimization of energy use mix. Inother example embodiments, the ML and/or AI for energy optimization mayinclude an ML and/or AI system configured for optimization ofproduction, storage, and utilization of a mix of energy sources andstorage elements (e.g., involving a process of production, storage, useof delivery system, and/or utilization).

In other example embodiments, the ML and/or AI for energy optimizationmay include and/or utilize systems and/or processes for providing energycost optimization across decentralized commerce models (e.g., an MLand/or AI system configured to optimize energy cost across decentralizedcommerce models). These ML and/or AI systems may be used with VCNs thatinclude several manufacturing locations and use multiple types of routesand types of transportation (e.g., third party logistics (3PL), fourthparty logistics (4PL), super grid logistics, logistics marketplaces).These ML and/or AI systems may optimize decisions on where/when tomanufacture and how/when to transport depending on real-time andpredicted cost of energy as an input. This may be accomplished bymonitoring all entities across the VCN and analyzing different variablesto ensure customer demands are being met while keeping energy costs to aminimum (e.g., using AI, predictive analytics, and quantum). The samemay be applied to multi-tenancy facilities to offer customersopportunities to optimize their energy use costs. In addition,predicting and securing future energy needs may be based on anticipatedcustomer demands.

In example embodiments, the energy system 19000 may include an ML/AIsystem configured for matching energy utilization and/or demand toenergy production across a distributed network (e.g., network of energyproduction, storage, and delivery systems).

In example embodiments, the energy system 19000 may include quantumfeatures. For example, these quantum features may include quantum foroptimizing energy utilization for a location, time, and/or application(e.g., a quantum computing system for optimizing energy utilization fora specific location, time window, and application). In exampleembodiments, the quantum computing system may address source productionof large quantities that may be timed to meet at a particular use window(e.g., shortly before asphalt is used) proximal to a point of use (e.g.,long haul may not be a cost-effective option), and on-site applicationmay require further energy to keep material pliable, deposit thematerial, and configure the material to prepare for use. On-siteapplication may also require several energy consuming devices (e.g.,spreaders, rollers, compressors) and materials (e.g., edge sealers,topcoat sealers, line striping). In example embodiments, decentralizedenergy systems may be configured proximal to target areas for asphaltuse (e.g., a new development, roadways such as for a town or a stretchof highway) that may need resurfacing. With flexible access to energyand storage, a construction crew may bring required (or excess demanded)energy with them as well.

In example embodiments, quantum features may generally include quantumcomputing. For example, quantum features may include quantumoptimization of power grids (e.g., a quantum computing system configuredto optimize a power grid). Examples of quantum computing may alsoinclude a quantum computing system configured for design optimization(e.g., design optimization of a battery) and/or a quantum computingsystem configured for real-time operating temperature optimization(e.g., operating temperature optimization of a battery). In exampleembodiments, quantum computing for design or real-time operatingtemperature optimization may include automated design and systemcontrol.

In other example embodiments, quantum features may include quantumbattery optimization such as quantum optimizing battery disposal (e.g.,a quantum computing system for optimizing battery disposal) and/orquantum optimizing battery recycling or reuse (e.g., a quantum computingsystem for optimizing battery recycling or reuse). In exampleembodiments, these quantum computing systems may address concernssurrounding supply chain optimization for primary and intermediatebattery materials which may be tied to more vertically integratedoperations including consolidation of processes and co-location ofbattery production and products. For example, these quantum computingsystems may address needs for battery manufacturing that may includematerial use optimized with VCN modelling. These quantum computingsystems may be utilized for battery manufacturing with 3d printedmaterials and processes. These quantum computing systems may also beutilized with data collection, management, and analysis that mayincorporate testing and tracking of battery “cell” and othersub-components for VCN optimization. The quantum computing systems maybe used with battery disposal, carbon footprint management, etc. as wellas for battery recycling or reuse (e.g., supply of lithium or cobaltneeded for lithium-ion batteries may come from recycled materials).

In example embodiments, quantum features may include quantumoptimization of energy use mix (e.g., a quantum computing system foroptimizing energy use mix). In other example embodiments, quantumfeatures may include energy cost optimization across decentralizedcommerce models (e.g., a quantum computing system configured to optimizeenergy cost across decentralized commerce models). These quantumcomputing systems may be used with VCNs that include severalmanufacturing locations and use multiple types of routes and types oftransportation (e.g., third party logistics (3PL), fourth partylogistics (4PL), super grid logistics, logistics marketplaces). Thesequantum computing systems may optimize decisions on where/when tomanufacture and how/when to transport depending on real-time andpredicted cost of energy as an input. This may be accomplished bymonitoring all entities across the VCN and analyzing different variablesto ensure customer demands are being met while keeping energy costs to aminimum (e.g., using AI, predictive analytics, and quantum). The samemay be applied to multi-tenancy facilities to offer customersopportunities to optimize their energy use costs. In addition,predicting and securing future energy needs may be based on anticipatedcustomer demands.

In example embodiments, the energy system 19000 may include technologiessuch as a system for slicing production, storage, and/or delivery ofenergy (e.g., type/mix, time, and location tracking).

In example embodiments, the energy system 19000 may be utilized in avariety of use cases. In example embodiments, the energy system orprocess 19000 may be applied with various use cases such as co-locationof modular/small-scale energy supply systems and various productionsystems for high-value items that may have localized demand (e.g.,growing high-margin foods, high-energy computational workloads, and/orhigh-temperature materials processes). In example embodiments, other usecases may include moving energy storage (e.g., a system having a networkof moving energy storage such as energy trucks), a food-energy valuechain network, fractional ownership of micro power stations (e.g., asystem for tokenizing ownership stakes in micro-power stations such thatthe investment costs of the micro-power station may be crowd-sourcedamongst owners), integration of solar panels and roadway (e.g., roadwayhaving integrated solar panels), a system for coordinating points ofsupply and demand intersection (e.g., both geographic and in time withland use permission, compute and/or data center resources, and energyavailability), battery-based printed circuit board fabrication plant(e.g., energy may be generated from renewable energy sources), and/orenergy index (e.g., an energy index to determine a price of goods). Insome example embodiments, robotics technology and energy optimizationtechnology may be utilized together (e.g., providing energy-optimizedplatform for autonomous robot operations).

In example embodiments, there may be various examples of fractionalownership of micro power stations. Ownership stakes in “micro-powerstations” may be tokenized (e.g., use of “ownership” tokens), such thatthe investment costs of such renewable energy sources may becrowd-sourced amongst owners. Each micro-power station may includerenewable energy sources, such as solar panels, wind turbines,geothermal energy conversion, and/or energy stores (e.g., largebatteries). Furthermore, a property owner on which the micro-powerstations may be located may also be awarded ownership tokens. A smartcontract may be created to release energy to a grid, whereby eachmicro-power station (e.g., energy source/battery) may be “coinoperated”. The smart contract may include conditional logic that maydefine a price for the energy and may be configured to instruct energystores/energy sources to release energy to the grid when the requisitefunds are deposited to an account associated with the smart contract.The smart contract may then apportion the funds to the collective ownersof the energy source, energy store, and/or property owner based on theownership tokens that each owner owns (e.g., owner of ¼ of the ownershiptokens for a micro-power station may receive ¼ of the distribution).Ownership tokens may then become tradeable, assuming a micro-powerstation is successful. While this contemplates putting energy back intothe “grid”, other examples may be applied to solutions relating toroadside vehicle chargers, industrial growth facilities, crypto miningfacilities, and/or the like. In some examples, a builder of a newfacility (e.g., an industrial growth, power plant, crypto-miningfacility) may crowd-source power generation without having to give upequity in their own business or taking out loans. In this scenario,excess energy may be sold into the grid, which would then increase thevalue of the ownership tokens.

In example embodiments, there may be various examples of a battery-basedprinted circuit board fabrication plant. For example, a printed circuitboard fabrication plant may receive raw materials, produce millions ofpolychlorinated biphenyls (PCBs) per day, have AI-enabled issue/defectidentification and prediction, and may ship the PCBs to ports,warehouses, distribution centers, etc. on electric vehicles (e.g.,electric trucks and cargo vans). The manufacturing plant may have solarpanels atop and nearby the plant building that may feed to batteriesstored within the plant. A nearby ocean coast may have wave energyharvesting devices installed, with the energy harvested therebytransmitted to the plant and stored in the batteries as well. A nearbyfield may have a wind farm stored thereon, with the wind energy beingtransmitted to the batteries. There may be several sets of batterieswhere each set of batteries may receive power from one or more of thesolar panels, wave energy harvesting devices, and wind farm. One of thesets of batteries may be configured to power the manufacturing plantduring operating hours. A second set of batteries, or a subset of thefirst set of batteries, may be configured to feed into transformersconfigured to charge the electric vehicles. The power coming from thesolar, wave, and wind energy sources may be conditioned for storage indifferent types of batteries and use in different applications, such asby adjusting phase, capacitance, etc. to feed electrical vehicle (EV)charging transformers vs. manufacturing machines, vs. high-poweredcomputing devices such as AI-enabled prediction and plantmanagement/control systems.

In example embodiments, there may be various examples of using an energyindex. For example, where this is competition on energy efficiency amongsuppliers in a private network or supply chain, an energy index may beadded to determine price of goods (e.g., uniform or comparable goodssuch as similar parts that may go into a final product). Goods that mayhave used more energy for manufacturing (or make and deliver) may besold at a discount while goods that may have used less energy formanufacturing (or make and deliver) may be sold at a premium. Ablockchain may be used to track batches, if not individual items. Theability to verify (or certify) may be necessary. The source of energymay also be a factor, such as coal vs. renewable. If multiplemanufacturers of a final product participate, anti-trust mechanisms mayneed to be in place. In some examples, these use cases may be tied withenergy production cost tracking technologies (e.g., to feed blockchain).

Dual Process Artificial Neural Networks

Referring to FIG. 104 in combination with FIG. 165 , in embodiments, theintelligence services include a dual process artificial neural network(DPANN) system 20000. The DPANN system 20000 includes an artificialneural network (ANN) having behaviors and operational processes (such asdecision-making) that are products of a training system and a retrainingsystem. The training system is configured to perform automatic, trainedexecution of ANN operations. The retraining system performs effortful,analytical, intentional retraining of the ANN, such as based on one ormore relevant aspects of the ANN, such as memory, one or more input datasets (including time information with respect to elements in such datasets), one or more goals or objectives (including ones that may varydynamically, such as periodically and/or based on contextual changes,such as ones relating to the usage context of the ANN), and/or others.In cases involving memory-based retraining, the memory may includeoriginal/historical training data and refined training data. The DPANNsystem 20000 includes a dual process learning function (DPLF) configuredto manage and perform an ongoing data retention process. The DPLF(including, where applicable, memory management process) facilitateretraining and refining of behavior of the ANN. The DPLF provides aframework by which the ANN creates outputs such as predictions,classifications, recommendations, conclusions and/or other outputs basedon a historic inputs, new inputs, and new outputs (including outputsconfigured for specific use cases, including ones determined byparameters of the context of utilization (which may include performanceparameters such as latency parameters, accuracy parameters, consistencyparameters, bandwidth utilization parameters, processing capacityutilization parameters, prioritization parameters, energy utilizationparameters, and many others).

In embodiments, the DPANN system 20000 stores training data, therebyallowing for constant retraining based on results of decisions,predictions, and/or other operations of the ANN, as well as allowing foranalysis of training data upon the outputs of the ANN. The management ofentities stored in the memory allows the construction and execution ofnew models, such as ones that may be processed, executed or otherwiseperformed by or under management of the training system. The DPANNsystem 20000 uses instances of the memory to validate actions (e.g., ina manner similar to the thinking of a biological neural network(including retrospective or self-reflective thinking about whetheractions that were undertaken under a given situation where optimal) andperform training of the ANN, including training that intentionally feedsthe ANN with appropriate sets of memories (i.e., ones that producefavorable outcomes given the performance requirements for the ANN).

In embodiments, the DPLF may be or include the continued processretention of one or more training datasets and/or memories stored in thememory over time. The DPLF thereby allows the ANN to apply existingneural functions and draw upon sets of past events (including ones thatare intentionally varied and/or curated for distinct purposes), such asto frame understanding of and behavior within present, recent, and/ornew scenarios, including in simulations, during training processes, andin fully operational deployments of the ANN. The DPLF may provide theANN with a framework by which the ANN may analyze, evaluate, and/ormanage data, such as data related to the past, present and future. Assuch, the DPLF plays a crucial role in training and retraining the ANNvia the training system and the retraining system.

In embodiments, the DPLF is configured to perform a dual-processoperation to manage existing training processes and is also configuredto manage and/or perform new training processes, i.e., retrainingprocesses. In embodiments, each instance of the ANN is trained via thetraining system and configured to be retrained via the retrainingsystem. The ANN encodes training and/or retraining datasets, stores thedatasets, and retrieves the datasets during both training via thetraining system and retraining via the retraining system. The DPANNsystem 20000 may recognize whether a dataset (the term dataset in thiscontext optionally including various subsets, supersets, combinations,permutations, elements, metadata, augmentations, or the like, relativeto a base dataset used for training or retraining), storage activity,processing operation and/or output, has characteristics that nativelyfavor the training system versus the retraining system based on itsrespective inputs, processing (e.g., based on its structure, type,models, operations, execution environment, resource utilization, or thelike) and/or outcomes (including outcome types, performance requirements(including contextual or dynamic requirements), and the like. Forexample, the DPANN system 20000 may determine that poor performance ofthe training system on a classification task may indicate a novelproblem for which the training of the ANN was not adequate (e.g., intype of data set, nature of input models and/or feedback, quantity oftraining data, quality of tagging or labeling, quality of supervision,or the like), for which the processing operations of the ANN are notwell-suited (e.g., where they are prone to known vulnerabilities due tothe type of neural network used, the type of models used, etc.), andthat may be solved by engaging the retraining system to retrain themodel to teach the model to learn to solve the new classificationproblem (e.g., by feeding it many more labeled instances of correctlyclassified items). With periodic or continuous evaluation of theperformance of the ANN, the DPANN system may subsequently determine thathighly stable performance of the ANN (such as where only smallimprovements of the ANN occur over many iterations of retraining by theretraining system) indicates readiness for the training system toreplace the retraining system (or be weighted more favorably where bothare involved). Over longer periods of time, cycles of varyingperformance may emerge, such as where a series of novel problems emerge,such that the retraining system of the DPANN is serially engaged, asneeded, to retrain the ANN and/or to augment the ANN by providing asecond source of outputs (which may be fused or combined with ANNoutputs to provide a single result (with various weightings acrossthem), or may be provided in parallel, such as enabling comparison,selection, averaging, or context- or situation-specific application ofthe respective outputs).

In embodiments, the ANN is configured to learn new functions inconjunction with the collection of data according to the dual-processtraining of the ANN via the training system and the retraining system.The DPANN system 20000 performs analysis of the ANN via the trainingsystem and performs initial training of the ANN such that the ANN gainsnew internal functions (or internal functions are subtracted ormodified, such as where existing functions are not contributing tofavorable outcomes). After the initial training, the DPANN system 20000performs retraining of the ANN via the retraining system. To perform theretraining, the retraining system evaluates the memory and historicprocessing of the ANN to construct targeted DPLF processes forretraining. The DPLF processes may be specific to identified scenarios.The ANN processes can run in parallel with the DPLF processes. By way ofexample, the ANN may function to operate a particular make and model ofa self-driving car after the initial training by the training system.The DPANN system 20000 may perform retraining of the functions of theANN via the retraining system, such as to allow the ANN to operate adifferent make and model of car (such as one with different cameras,accelerometers and other sensors, different physical characteristics,different performance requirements, and the like), or even a differentkind of vehicle, such as a bicycle or a spaceship.

In embodiments, as quality of outputs and/or operations of the ANNimproves, and as long as the performance requirements and the context ofutilization for the ANN remain fairly stable, performing thedual-process training process can become a decreasingly demandingprocess. As such, the DPANN system 20000 may determine that fewerneurons of the ANN are required to perform operations and/or processesof the ANN, that performance monitoring can be less intensive (such aswith longer intervals between performance checks), and/or that theretraining is no longer necessary (at least for a period of time, suchas until a long-term maintenance period arrives and/or until there aresignificant shifts in context of utilization). As the ANN continues toimprove upon existing functions and/or add new functions via thedual-process training process, the ANN may perform other, at times more“intellectually-demanding” (e.g., retraining intensive) taskssimultaneously. For example, utilizing dual process-learned knowledge ofa function or process being trained, the ANN can solve an unrelatedcomplex problem or make a retraining decision simultaneously. Theretraining may include supervision, such as where an agent (e.g., humansupervisor or intelligent agent) directs the ANN to a retrainingobjective (e.g., “master this new function”) and provides a set oftraining tasks and feedback functions (such as supervisory grading) forthe retraining. In-embodiments, the ANN can be used to organize thesupervision, training and retraining of other dual process-trained ANNs,to seed such training or retraining, or the like.

In embodiments, one or more behaviors and operational processes (such asdecision-making) of the ANN may be products of training and retrainingprocesses facilitated by the training system and the retraining system,respectively. The training system may be configured to perform automatictraining of ANN, such as by continuously adding additional instances oftraining data as it is collected by or from various data sources. Theretraining system may be configured to perform effortful, analytical,intentional retraining of the ANN, such as based on memory (e.g., storedtraining data or refined training data) and/or optionally based onreasoning or other factors. For example, in a deployment managementcontext, the training system may be associated with a standard responseby the ANN, while the retraining system may implement DPLF retrainingand/or network adaptation of the ANN. In some cases, retraining of theANN beyond the factory, or “out-of-the-box,” training level may involvemore than retraining by the retraining system. Successful adjustment ofthe ANN by one or more network adaptations may be dependent on theoperation of one or more network adjustments of the training system.

In embodiments, the training system may facilitate fast operating by andtraining of the ANN by applying existing neural functions of the ANNbased on training of the ANN with previous datasets. Standardoperational activities of the ANN that may draw heavily on the trainingsystem may include one or more of the methods, processes, workflows,systems, or the like described throughout this disclosure and thedocuments incorporated herein, such as, without limitation: definedfunctions within networking (such as discovering available networks andconnections, establishing connections in networks, provisioning networkbandwidth among devices and systems, routing data within networks,steering traffic to available network paths, load balancing acrossnetworking resources, and many others); recognition and classification(such as of images, text, symbols, objects, video content, music andother audio content, speech content, and many others); spoken words;prediction of states and events (such as prediction of failure modes ofmachines or systems, prediction of events within workflows, predictionsof behavior in shopping and other activities, and many others); control(such as controlling autonomous or semi-autonomous systems, automatedagents (such as automated call-center operations, chat bots, and thelike) and others); and/or optimization and recommendation (such as forproducts, content, decisions, and many others). ANNs may also besuitable for training datasets for scenarios that only require output.The standard operational activities may not require the ANN to activelyanalyze what is being asked of the ANN beyond operating on well-defineddata inputs, to calculate well-defined outputs for well-defined usecases. The operations of the training system and/or the retrainingsystem may be based on one or more historic data training datasets andmay use the parameters of the historic data training datasets tocalculate results based on new input values and may be performed withsmall or no alterations to the ANN or its input types. In embodiments,an instance of the training system can be trained to classify whetherthe ANN is capable of performing well in a given situation, such as byrecognizing whether an image or sound being classified by the ANN is ofa type that has historically been classified with a high accuracy (e.g.,above a threshold).

In embodiments, network adaptation of the ANN by one or both of thetraining system and the retraining system may include a number ofdefined network functions, knowledge, and intuition-like behavior of theANN when subjected to new input values. In such embodiments, theretraining system may apply the new input values to the DPLF system toadjust the functional response of the ANN, thereby performing retrainingof the ANN. The DPANN system 20000 may determine that retraining the ANNvia network adjustment is necessary when, for example, withoutlimitation, functional neural networks are assigned activities andassignments that require the ANN to provide a solution to a novelproblem, engage in network adaptation or other higher-order cognitiveactivity, apply a concept outside of the domain in which the DPANN wasoriginally designed, support a different context of deployment (such aswhere the use case, performance requirements, available resources, orother factors have changed), or the like. The ANN can be trained torecognize where the retraining system is needed, such as by training theANN to recognize poor performance of the training system, highvariability of input data sets relative to the historical data sets usedto train the training system, novel functional or performancerequirements, dynamic changes in the use case or context, or otherfactors. The ANN may apply reasoning to assess performance and providefeedback to the retraining system. The ANN may be trained and/orretrained to perform intuitive functions, optionally including by acombinatorial or re-combinatorial process (e.g., including geneticprogramming wherein inputs (e.g., data sources), processes/functions(e.g., neural network types and structures), feedback, and outputs, orelements thereof, are arranged in various permutations and combinationsand the ANN is tested in association with each (whether in simulationsor live deployments), such as in a series of rounds, or evolutionarysteps, to promote favorable variants until a preferred ANN, or preferredset of ANNs is identified for a given scenario, use case, or set ofrequirements). This may include generating a set of input “ideas” (e.g.,combinations of different conclusions about cause-and-effect in adiagnostic process) for processing by the retraining system andsubsequent training and/or by an explicit reasoning process, such as aBayesian reasoning process, a casuistic or conditional reasoningprocess, a deductive reasoning process, an inductive reasoning process,or others (including combinations of the above) as described in thisdisclosure or the documents incorporated herein by reference.

Referring to FIG. 2XM, in embodiments, the DPLF may perform an encodingprocess of the DPLF to process datasets into a stored form for futureuse, such as retraining of the ANN by the retraining system. Theencoding process enables datasets to be taken in, understood, andaltered by the DPLF to better support storage in and usage from thememory. The DPLF may apply current functional knowledge and/or reasoningto consolidate new input values. The memory can include short-termmemory (STM), long-term memory (LTM), or a combination thereof. Thedatasets may be stored in one or both of the STM and the LTM. The STMmay be implemented by the application of specialized behaviors insidethe ANN (such as recurrent neural network, which may be gated orun-gated, or long-term short-term neural networks). The LTM may beimplemented by storing scenarios, associated data, and/or unprocesseddata that can be applied to the discovery of new scenarios. The encodingprocess may include processing and/or storing, for example, visualencoding data (e.g., processed through a Convolution Neural Network),acoustic sensor encoding data (e.g., how something sounds, speechencoding data (e.g., processed through a deep neural network (DNN),optionally including for phoneme recognition), semantic encoding data ofwords, such to determine semantic meaning, e.g., by using a HiddenMarkov Model (HMM); and/or movement and/or tactile encoding data (suchas operation on vibration/accelerometer sensor data, touch sensor data,positional or geolocation data, and the like). While datasets may enterthe DPLF system through one of these modes, the form in which thedatasets are stored may differ from an original form of the datasets andmay pass-through neural processing engines to be encoded into compressedand/or context-relevant format. For example, an unsupervised instance ofthe ANN can be used to learn the historic data into a compressed format.

In embodiments, the encoded datasets are retained within the DPLFsystem. Encoded datasets are first stored in short-term DPLF, i.e., STM.For example, sensor datasets may be primarily stored in STM, and may bekept in STM through constant repetition. The datasets stored in the STMare active and function as a kind of immediate response to new inputvalues. The DPANN system 20000 may remove datasets from STM in responseto changes in data streams due to, for example, running out of space inSTM as new data is imported, processed and/or stored. For example, it isviable for short-term DPLF to only last between 15 and 30 seconds. STMmay only store small amounts of data typically embedded inside the ANN.

In embodiments, the DPANN system 20000 may measure attention based onutilization of the training system, of the DPANN system 20000 as awhole, and/or the like, such as by consuming various indicators ofattention to and/or utilization of outputs from the ANN and transmittingsuch indicators to the ANN in response (similar to a “moment ofrecognition” in the brain where attention passes over something and thecognitive system says “aha!”). In embodiments, attention can be measuredby the sheer amount of the activity of one or both of the systems on thedata stream. In embodiments, a system using output from the ANN canexplicitly indicate attention, such as by an operator directing the ANNto pay attention to a particular activity (e.g., to respond to adiagnosed problem, among many other possibilities). The DPANN system20000 may manage data inputs to facilitate measures of attention, suchas by prompting and/or calculating greater attention to data that hashigh inherent variability from historical patterns (e.g., in rates ofchange, departure from norm, etc.), data indicative of high variabilityin historical performance (such as data having similar characteristicsto data sets involved in situations where the ANN performed poorly intraining), or the like.

In embodiments, the DPANN system 20000 may retain encoded datasetswithin the DPLF system according to and/or as part of one or morestorage processes. The DPLF system may store the encoded datasets in LTMas necessary after the encoded datasets have been stored in STM anddetermined to be no longer necessary and/or low priority for a currentoperation of the ANN, training process, retraining process, etc. The LTMmay be implemented by storing scenarios, and the DPANN system 20000 mayapply associated data and/or unprocessed data to the discovery of newscenarios. For example, data from certain processed data streams, suchas semantically encoded datasets, may be primarily stored in LTM. TheLTM may also store image (and sensor) datasets in encoded form, amongmany other examples.

In embodiments, the LTM may have relatively high storage capacity, anddatasets stored within LTM may, in some scenarios, be effectively storedindefinitely. The DPANN system 20000 may be configured to removedatasets from the LTM, such as by passing LTM data through a series ofmemory structures that have increasingly long retrieval periods orincreasingly high threshold requirements to trigger utilization (similarto where a biological brain “thinks very hard” to find precedent to dealwith a challenging problem), thereby providing increased salience ofmore recent or more frequently used memories while retaining the abilityto retrieve (with more time/effort) older memories when the situationjustifies more comprehensive memory utilization. As such, the DPANNsystem 20000 may arrange datasets stored in the LTM on a timeline, suchas by storing the older memories (measured by time of origination and/orlatest time of utilization) on a separate and/or slower system, bypenalizing older memories by imposing artificial delays in retrievalthereof, and/or by imposing threshold requirements before utilization(such as indicators of high demand for improved results). Additionallyor alternatively, LTM may be clustered according to other categorizationprotocols, such as by topic. For example, all memories proximal in timeto a periodically recognized person may be clustered for retrievaltogether, and/or all memories that were related to a scenario may beclustered for retrieval together.

In embodiments, the DPANN system 20000 may modularize and link LTMdatasets, such as in a catalog, a hierarchy, a cluster, a knowledgegraph (directed/acyclic or having conditional logic), or the like, suchas to facilitate search for relevant memories. For example, all memorymodules that have instances involving a person, a topic, an item, aprocess, a linkage of n-tuples of such things (e.g., all memory modulesthat involve a selected pair of entities), etc. The DPANN system 20000may select sub-graphs of the knowledge graph for the DPLF to implementin one or more domain-specific and/or task-specific uses, such astraining a model to predict robotic or human agent behavior by usingmemories that relate to a particular set of robotic or human agents,and/or similar robotic or human agents. The DPLF system may cachefrequently used modules for different speed and/or probability ofutilization. High value modules (e.g., ones with high-quality outcomes,performance characteristics, or the like) can be used for otherfunctions, such as selection/training of STM keep/forget processes.

In embodiments, the DPANN system 20000 may modularize and link LTMdatasets, such as in various ways noted above, to facilitate search forrelevant memories. For example, memory modules that have instancesinvolving a person, a topic, an item, a process, a linkage of n-tuplesof such things (such as all memory modules that involve a selected pairof entities), or all memories associated with a scenario, etc., may belinked and searched. The DPANN system 20000 may select subsets of thescenario (e.g., sub-graphs of a knowledge graph) for the DPLF for adomain-specific and/or task-specific use, such as training a model topredict robotic or human agent behavior by using memories that relate toa particular set of robotic or human agents and/or similar robotic orhuman agents. Frequently used modules or scenarios can be cached fordifferent speed/probability of utilization, or other performancecharacteristics. High value modules or scenarios (ones wherehigh-quality outcomes results) can be used for other functions, such asselection/training of STM keep/forget processes, among others.

In embodiments, the DPANN system 20000 may perform LTM planning, such asto find a procedural course of action for a declaratively describedsystem to reach its goals while optimizing overall performance measures.The DPANN system 20000 may perform LTM planning when, for example, aproblem can be described in a declarative way, the DPANN system 20000has domain knowledge that should not be ignored, there is a structure toa problem that makes the problem difficult for pure learning techniques,and/or the ANN needs to be trained and/or retrained to be able toexplain a particular course of action taken by the DPANN system 20000.In embodiments, the DPANN system 20000 may be applied to a planrecognition problem, i.e., the inverse of a planning problem: instead ofa goal state, one is given a set of possible goals, and the objective inplan recognition is to find out which goal was being achieved and how.

In embodiments, the DPANN system 20000 may facilitate LTM scenarioplanning by users to develop long-term plans. For example, LTM scenarioplanning for risk management use cases may place added emphasis onidentifying extreme or unusual, yet possible, risks and opportunitiesthat are not usually considered in daily operations, such as ones thatare outside a bell curve or normal distribution, but that in fact occurwith greater-than-anticipated frequency in “long tail” or “fat tail”situations, such as involving information or market pricing processes,among many others. LTM scenario planning may involve analyzingrelationships between forces (such as social, technical, economic,environmental, and/or political trends) in order to explain the currentsituation, and/or may include providing scenarios for potential futurestates.

In embodiments, the DPANN system 20000 may facilitate LTM scenarioplanning for predicting and anticipating possible alternative futuresalong with the ability to respond to the predicted states. The LTMplanning may be induced from expert domain knowledge or projected fromcurrent scenarios, because many scenarios (such as ones involvingresults of combinatorial processes that result in new entities orbehaviors) have never yet occurred and thus cannot be projected byprobabilistic means that rely entirely on historical distributions. TheDPANN system 20000 may prepare the application to LTM to generate manydifferent scenarios, exploring a variety of possible futures to the DPLMfor both expected and surprising futures. This may be facilitated oraugmented by genetic programming and reasoning techniques as notedabove, among others.

In embodiments, the DPANN system 20000 may implement LTM scenarioplanning to facilitate transforming risk management into a planrecognition problem and apply the DPLF to generate potential solutions.LTM scenario induction addresses several challenges inherent to forecastplanning. LTM scenario induction may be applicable when, for example,models that are used for forecasting have inconsistent, missing,unreliable observations; when it is possible to generate not just onebut many future plans; and/or when LTM domain knowledge can be capturedand encoded to improve forecasting (e.g., where domain experts tend tooutperform available computational models). LTM scenarios can be focusedon applying LTM scenario planning for risk management. LTM scenariosplanning may provide situational awareness of relevant risk drivers bydetecting emerging storylines. In addition, LTM scenario planning cangenerate future scenarios that allow DPLM, or operators, to reasonabout, and plan for, contingencies and opportunities in the future.

In embodiments, the DPANN system 20000 may be configured to perform aretrieval process via the DPLF to access stored datasets of the ANN. Theretrieval process may determine how well the ANN performs with regard toassignments designed to test recall. For example, the ANN may be trainedto perform a controlled vehicle parking operation, whereby theautonomous vehicle returns to a designated spot, or the exit, byassociating a prior visit via retrieval of data stored in the LTM. Thedatasets stored in the STM and the LTM may be retrieved by differingprocesses. The datasets stored in the STM may be retrieved in responseto specific input and/or by order in which the datasets are stored,e.g., by a sequential list of numbers. The datasets stored in the LTMmay be retrieved through association and/or matching of events tohistoric activities, e.g., through complex associations and indexing oflarge datasets.

In embodiments, the DPANN system 20000 may implement scenario monitoringas at least a part of the retrieval process. A scenario may providecontext for contextual decision-making processes. In embodiments,scenarios may involve explicit reasoning (such as cause-and-effectreasoning, Bayesian, casuistic, conditional logic, or the like, orcombinations thereof) the output of which declares what LTM-stored datais retrieved (e.g., a timeline of events being evaluated and othertimelines involving events that potentially follow a similarcause-and-effect pattern). For example, diagnosis of a failure of amachine or workflow may retrieve historical sensor data as well as LTMdata on various failure modes of that type of machine or workflow(and/or a similar process involving a diagnosis of a problem state orcondition, recognition of an event or behavior, a failure mode (e.g., afinancial failure, contract breach, or the like), or many others).

Edge-Distributed Database and Edge Query Language

FIGS. 166-172 relate to various embodiments of a distributed databaseand query language that is configured to store and retrieve a widevariety of data, including data generated by various components of avalue chain network as described herein. In some embodiments, thedistributed database may be configured to store data at variouscomponents of a network, which may include any of the networks describedthroughout the disclosure. As will be discussed, these components mayact as edge devices and/or aggregators to store data in a scalable anddistributed manner. Although devices are described as being edge devicesand/or aggregators, it should be understood that the edge devices and/oraggregators may be implemented by any of the networked devices describedthroughout this disclosure. Furthermore, it should be understood thatthe data to be stored in the distributed database may be any of the datadescribed throughout this disclosure, including (without limitation)value chain data, sensor data generated by any of the sensors describedherein, social data, product data, service data, management data, riskdata, distributed ledger data, database data, various data generated byartificial intelligence components, analysis components, machinelearning components, and/or any other data as described herein.

Techniques described herein improve the ability of networks and systemsto deal with large volumes of data on edge devices by leveraging thestorage and processing capabilities of the edge devices to provide adistributed database system and a query language for efficientlyquerying the distributed database system. According to techniquesdescribed herein, a database layer of an application stack can bedistributed across all nodes of a network including the edge nodes suchthat vast amounts of data may be stored locally at these nodes toprovide access to the data in response to a query. In such a distributeddatabase environment, queries may be received and/or executed by edgedistributed node points so that results may be provided quickly andsecurely. According to techniques described herein, the entire networkenvironment may appear as a seamless database, and an Edge QueryLanguage (referring to herein as “EDQL”) may provide for resolution ofthe query.

Accordingly to techniques described herein, capabilities that wereformerly located in the cloud may be extended to the edge environment inorder to provide a seamless services infrastructure extending from thecloud to edge components.

According to techniques described herein, services running on the edgemay be stateless, which may allow for them to be dynamically movedbetween different physical devices without having to considerconfiguration parameters. Data may be housed dynamically andtransitioned seamlessly between various edge nodes and supporting nodes(e.g., aggregators). The resulting data environment may shard datadynamically, allowing for EDQL users to query the data seamlessly (e.g.,at edge nodes) from any location.

In embodiments, each edge component may be part of a microservicesinfrastructure that allows for seamless distribution of applicationlogic and data processing. The underlying EDQL components may be looselycoupled and dynamically deployed in response to query workload. Adynamic ledger holding core data location and probabilistic distributionmodels may be used to allow for data to be dynamically queried acrossthe entire network (e.g., IoT network) and for the correct microservicescomponents to respond to the queries. In addition, the dynamic ledgermay allow for query results to be approximated based on probabilitytables in order to provide results that are within an accepted margin oferror.

Many applications require analytical processing as an intrinsic part oftheir operational framework such that the framework requires deep accessto multiple data sets. According to techniques described herein, each ofthe data sets in such frameworks may exist in variable levels of grain.For example, data may be summarized and distributed at different grainlevels across the edge network. By dynamically allocating data andbuilding probability distribution models in response to queries to anedge device, edge queries can be executed, and responses generated,without having to centralize data.

Edge environments may require extremely low latency for responses toqueries (e.g., single digit milliseconds). This requirement may also becombined with a requirement for fine grained data response services(e.g., seeing specific events at the finest grain). According totechniques described herein, either or both of these requirements may behandled by configuring edge nodes to collect and provision vastquantities of data without the need for massive data flows to acentralized and consolidated database.

In embodiments, activities that are closer to a control system mayrequire much finer data grains and faster response times. In addition,edge nodes may not require aggregation of data (e.g., if they areresponding in the moment to specific events). According to techniquesdescribed herein, EDQL leverages these usage patterns to allow fordistributed queries to the edge against the entire edge-shardeddatabase, where fine grained data is held in high volume edge systemsand transmitted as the EDQL system deems the data required.

Techniques described herein provide several technological benefits whencompared to prior database solutions. First, distributed databasesconfigured as described herein provide for powerful and seamlessabstraction of data queries from the underlying data structures, suchthat query users do not need to worry about the edge-distributedstorage. Additionally, distributed databases configured as describedherein provide for seamless distribution and management of a dynamicledger of information relating to underlying datasets. Additionally, theEDQL database provides for potential failure rate on queries to allowfor prioritization of results.

In embodiments, each edge node and/or edge cluster can hold dataindependently and redundantly, making the database fault tolerant tocentralized failure. Additionally, localized encryption and/or blockchain storage mechanisms may ensure localized data is secure againstcyber-attack in both nature and type.

In embodiments, distributed databases as configured herein optimizenetwork usage such that, rather than burying the system in massivevolumes of data, systems configured as described herein may focus ontransmitting data that is required to respond to a received query orpredicted future query. Furthermore, centralized queries againstdistributed fine-grained data may be possible, thus providing access tothe finest grain data without overloading a network.

In embodiments, EDQL may be implemented using extensions of a structuredquery language (SQL), such as by using data definition language (DDL)and/or data manipulation language (DML) extensions of SQL. Inparticular, DDL and DML extensions may extend SQL to handle thedistribution of data across the distributed database. The use of SQL andextensions thereof may make it easy to query data, whether in in memory,rowstore or columnstore tables, with a well-understood language extendedusing DDL and DML extensions.

In embodiments, every edge table DDL may have at least one edge shardalgorithm, which can contain any number of column parameters. The shardalgorithms may be used to distribute data across the network.

In embodiments, the distributed database may be configured toefficiently execute any join query, taking advantage of opportunities toimprove efficiency based on edge sharding and replicated referencetables. Because reference tables may be replicated on some or alldevices in a cluster, edge nodes can join against local copies ofreference tables with optimal performance.

In embodiments, edge query optimizers may leverage edge shard algorithmsto determine how a query should be executed. For example, queries thatfully match an edge shard algorithm can be routed directly to a singlepartition associated with a single edge device. For queries that need toshuffle data across nodes, data movement may be minimized through theuse of probabilistic data distributions described in a dynamic ledger.

In embodiments, data may be duplicated at various grain levels throughthe use of a dynamic ledger and probabilistic models, which allow fortargeted and potentially massive distribution of data while leveraginglower storage costs to provide for potentially massive distributedscalability.

In embodiments, the distributed database system may allow for dataupdates and versioning while handling the potential cascading impacts ofa single update. The distributed database system may allow for multipleversions of the truth based on latency (with all data also knowing itsstate as at a point in time).

In embodiments, techniques described herein provide the ability todistribute and process data evenly across all the edge nodes in adistributed cluster, thus supporting horizontal scale-out (e.g., of anIoT platform-based data collection system) with considerably less needto move data to centralized systems. This effect reduces networkrequirements and allows for greater scalability of the final system.

In embodiments, edge nodes added to the system may primarily store dataaccording to the localized needs of nearby edge systems (e.g., edgesystems connected via local networks or other high bandwidth and/or lowlatency links), which may allow for additional nodes to be added withoutimpairing the operation of a centralized monitoring system.

In embodiments, edge nodes connected to a probability-based dynamicledger and localized storage are able to operate without centralconnectivity. This independence allows for greatly increased faulttolerance and removes dependence on network communications for provisionof highly available systems.

Other distributed systems tend to operate on NoSQL (“not only SQL”)environments where data is stored in a form that is very close to theinput format. This centralized format forces the application developerto have a direct understanding of the fundamental data and indexstructures. Systems and techniques described herein, by contrast, offerscale-out benefits and (in some embodiments) may also provideconsistency and a SQL-like interface. Additionally, systems andtechniques described herein offer an advantage for new, modern,edge-enabled cloud-native applications by providing a seamless dataaccess layer that is similar to the underlying layer in most applicationframeworks.

FIG. 166A shows an environment 20100 including a plurality of devicesthat implement and/or interact with the distributed database and dynamicledger as discussed herein. The environment 2020100 may include one ormore query devices 20110 that may generate queries for querying thedistributed database as further described herein. The environment 20100may further include one or more edge devices 20120 comprising and/orconnected to edge storage 20122, which may store data that may be usedto respond to queries received from query devices. In embodiments, theenvironment 20100 may further include one or more aggregators 20140,which may communicate with and/or implement a dynamic ledger 20150 thatmay be used to store “high grain” data (also referred to herein as“summary” data) based on data stored at various edge devices, toimplement various probabilistic models for the generation of queryresponses, to distributed data throughout the distributed database,and/or to perform other dynamic ledger functions as described herein. Insome embodiments, edge device/aggregators 20130 may perform thefunctions of both edge devices 20120 and/or aggregators 20140, and thusmay include and/or communicate with various edge storage 20132,communicate with and/or implement a dynamic ledger 20150, and otherwiseperform the functions ascribed to edge devices 20120 and/or thefunctions ascribed to aggregators 20140 as discussed herein. Althoughthe environment 20100 illustrates edge devices 20120, edgedevices/aggregators 20130, and aggregators 20140, in some embodimentsnot all of these devices may be used. For example, a particularembodiment may use edge devices 20120 and aggregators 20140, but notedge devices/aggregators 20130. Additionally or alternatively, aparticular embodiment may use edge devices/aggregators 20130, but notedge devices 20120 or aggregators 20140.

In embodiments, the distributed database may include multiple clusters(not shown), each of which may include a plurality of edge devices20120, aggregators 20140, and/or edge/device aggregators 20130. Forexample, a first cluster may correspond to various nodes/devices in afirst location, while a second cluster may correspond to variousnodes/devices in a second location.

In embodiments, query devices 20110 may be any computing device that maybe capable of generating and transmitting a query. In some embodiments,the distributed database may store, for example, sensor data captured byvarious sensors that are part of and/or in communication with edgedevices. In these embodiments, the query devices 20110 may thus includeany device that wishes to obtain sensor data, summary data generatedbased on sensor data, probabilistic data generated based on sensor data,and/or the like. The query devices 20110 in these embodiments mayinclude various control systems, monitoring systems, user devices (e.g.,a device associated with a maintenance engineer tasked with monitoringthe system and/or diagnosing problems), prediction systems (e.g., adevice tasked with predicting a future state based on current or pastsensor data), security systems, customer systems, supplier systems,and/or the like. However, it should be noted that sensor data is merelyan example type of data that may be stored in the distributed database,and the distributed database may therefore store other types of datathat may be useful for other applications.

In embodiments, edge devices 20120 (and/or edge devices/aggregators20130) may include and/or communicate with a sensor or other data sourcethat generates data for storage in the distributed database (e.g., viaedge storage). Edge devices thus may be responsible for maintaining thedata generated by the sensor or other data source in the edge storage.As discussed above, edge devices may maintain very large volumes of datain edge storage, such that it may be impractical or impossible tocentralize all of the edge data. Accordingly, edge devices may beconfigured to provide limited amounts of data (e.g., slices of edgedata, summary data based on edge data, parameters for probabilisticmodels that describe edge data, reference tables based on edge data,etc.) to other devices in the network and/or to a dynamic ledger, asdescribed in more detail herein. In embodiments, an edge device mayreceive a query from a query device, determine a query plan forobtaining any necessary data and responding to the query, causeexecution of the query plan, and provide a query response to the querydevice 20110. In other words, a query device 20110 may transmit a queryto an edge device 20120, which may be configured to handle the queryusing the techniques described herein.

In embodiments, edge devices 20120 and/or aggregators 20140 (e.g.,including edge devices/aggregators 20130) may communicate with and/orimplement a dynamic ledger 20150 that may store various data forenabling and optimizing the distributed network. As discussed herein,data stored in edge storage may be too voluminous to be centrallystored, and thus the aggregators may maintain a dynamic ledger that mayinstruct edge notes to move, summarize, and/or store summary data thatmay be used to respond to certain queries, probabilistic models built byedge nodes that may be used to respond to certain queries, and otherdata that may be used to formulate query responses without requiringprohibitively large amounts of network traffic to and from various edgedevices. In some embodiments, the dynamic ledger 20150 may be ablockchain, and in these embodiments, the aggregators may be blockchainnodes that may be used to “mine” new blocks, distribute new blocks toother nodes (e.g., edge nodes), implement consensus algorithms, and/orthe like. Additionally or alternatively, the dynamic ledger 20150 may bea ledger that is not a blockchain, and the aggregators may use varioustechniques and/or architectures as described herein to create dynamicledger instructions for the edge nodes to share the data stored on thedynamic ledger 20150.

In embodiments, an aggregator 20140 may receive a query from a querydevice, determine a query plan for obtaining any necessary data andresponding to the query, cause execution of the query plan on the edgenode, and provide a query response to the query device 20110. In otherwords, a query device 20110 may transmit a query to an aggregator, whichmay be configured to handle the query using the techniques describedherein. In embodiments, users may interact with an aggregator 20140 oran edge node as if it were the database, running queries and updatingdata as normal via query commands (e.g., EDQL commands). In response,the aggregator 20140 may create instructions on the dynamic ledger forthe edge nodes to execute queries, aggregate intermediate results, andsend final results back to the query device. Communication betweenaggregators and edge nodes for query execution may be implemented asEDQL statements.

In embodiments, aggregators 20140 may operate as load balancers and/ornetwork proxies through which query devices may interact with a clusterof the distributed database. For example, aggregators 20140 may createdynamic ledger instructions to cause data to be shifted between edgedevices (e.g., replicated from edge storage associated with a first edgedevice to edge storage associated with a second edge device) in order tooptimize the performance of the network (e.g., by moving data closer todevices that are receiving queries for that data).

In embodiments, data may be sharded across the edge devices intopartitions. The number of partitions may be configurable on a clusterlevel with a set variable and/or may be available as an optionalparameter (e.g., to a DDL statement). Additionally or alternatively, thenumber of partitions may be based on usage patterns rather than hardcoded column-based partition names. In the context of query execution, apartition may be the granular unit of query parallelism. In someembodiments, every parallel query is run with a level of parallelismequal to the number of partitions. In others, an additional degree ofparallelism is provided which is intra-partition parallelism.

In embodiments, the various devices may communicate using one or morenetworks 20160, which may include the Internet and/or othercommunication networks. In some embodiments, the various devices thatimplement and/or interact with the distributed database may be separatedby large distances, and thus may communicate via various local networksas well as wide area networks. For example, a distributed database forstoring sensor data may include edge devices 20120 in various cities,states, countries, or other locations, all of which may communicate viavarious networks 20160 to implement the functions and features describedherein.

FIG. 166B illustrates an example architecture 20170 for connectingvarious devices in a mesh network configuration. Although thearchitecture 20170 shows a few devices, in practice the number of edgedevices, aggregators, and/or edge device/aggregators may be much largerin number. Additionally or alternatively, instead of using a meshnetwork architecture, the devices may use a fully connected architectureor some other type of network architecture. As shown in FIG. 166B, aquery device 20110 may connect to one or more edge devices (e.g., edgedevice 20120B, edge device/aggregator 20130B), which provide APIs and/orother interfaces for receiving queries from query devices 20110. Forexample, the query device 20110 may connect to edge device 20120B, whichmay store a first set of edge data at edge storage connected to edgedevice 20120B, and/or to edge device/aggregator 20130B, which may storea second set of edge data at edge storage connected to edgedevice/aggregator 20130B. In embodiments, although the edge devices maystore different edge data in edge storage, using techniques describedherein, they may provide the same or similar responses to some queries(e.g., queries that are not specific to an edge device). For example,whether the query device 20110 sends a particular query (e.g., a queryfor an average sensor reading for a particular region) to a first edgedevice in the region or a second edge device in the region, the queryresponse may be the same (or statistically similar). In this example,the edge devices may provide the same or similar responses because theymay access and/or use the same summary data and/or probabilistic modelsstored on the dynamic ledger 20150 to respond to the query.

In embodiments, the query device 20110 may additionally or alternativelyconnect directly to (and/or send queries directly to) various edgedevices/aggregators 20130 and/or aggregators 20140. In embodiments, theedge devices 20120, edge devices/aggregators 20130, and/or aggregators20140 may connect directly to the dynamic ledger 20150 and/or mayconnect via other devices to the dynamic ledger 20150. In embodiments,the network may maintain multiple dynamic ledgers 20150 that may be usedto store different types of data (e.g., a first dynamic ledger forstoring a first type of data and a second dynamic ledger for storing asecond type of data) and/or data that may be used for differentpurposes.

FIG. 167A illustrates details of example data stored in edge storage20122, which may be connected to various edge devices, which in turn mayinclude and/or be connected to various sensors 20202 and/or other datasources 20204. In embodiments, to leverage storage across independentstorage devices, the database may be replicated and/or edge sharded(typically in part) across multiple edge devices by storing various datain different partitions, where each partition may correspond to adifferent edge storage 20122. This sharding and/or replication enablesthe database to execute query fragments on each partition and thencombine the results to produce a single answer. Applications and/orusers may have no knowledge of where data is physically located or howtables are partitioned and/or replicated. A partitioning scheme (asdescribed in more detailed below) may maximize single edge nodetransactions to avoid the need to coordinate the behavior of concurrenttransactions running on other nodes.

As shown in FIG. 167A, each edge device 20120 may store detailed datafrom connected sensors 20202 and/or other data sources 20204 inconnected edge storage 20122. As shown, an example edge device 20120Amay include and/or be connected to sensors 20202A and/or data sources20204A. Correspondingly, another example edge device 20120B may includeand/or be connected to sensors 20202B and/or data sources 20204B. Theedge device 20120A may continually receive data from the sensors 20202Aand/or other data sources 20204A, and the edge device 20120B maycontinually receive data from the sensors 20202B and/or other datasources 20204B. The sensors may be any types of sensors, includingenvironmental sensors (e.g., temperature sensors, weather sensors),visual sensors (e.g., image and/or video cameras), audio sensors (e.g.,microphones and/or acoustic sensors), location/orientation sensors(e.g., accelerometers, gyroscopes, speedometers, GPS chips, etc.),vibration sensors, chemical sensors, biological sensors, or any otherform of sensor. Moreover, the data sources may include any type of datasource that produces data that may be stored in a distributed database,such as devices that generate various reports or analyses, devices thatmonitor the status of networks or network devices, security devices,devices that monitor production lines, devices that monitor traffic,and/or any other types of data sources. The edge storage 20122A, 20122Bmay store detailed data (e.g., sensors A detailed data 20212A and/ordata sources A detailed data 20214A in edge storage 20122A, and sensorsB detailed data 20212B and/or data sources B detailed data 20214B inedge storage 20122B) collected over long periods of time by the sensors20202 and/or data sources 20204, such as continuous streams of sensorreadings collected over a long period of time, periodicreports/analyses/figures/etc. generated by the data sources over longperiods of time, and/or the like. In embodiments, the edge storage mayinclude megabytes, gigabytes, terabytes, or even more of detailed datacollected by the edge device 20120 and stored in edge storage 20122. Inembodiments, the data may be too voluminous to share across thedistributed database, and therefore the edge devices must be configuredto enable responses to high volumes of queries without continuouslytransmitting large amounts of detailed data across a network connectingthe various components of the distributed database. In embodiments, thisresult may be achieved through the use of dynamic ledger instructions.The aggregator nodes can monitor the dynamic ledger and create furtherinstructions to optimize the location of the data or summaries of thedata.

In some embodiments, detailed data may be stored redundantly in edgestorage. For example, in the illustrated embodiment edge storage 20122Bfurther includes redundant data 20222B, 20224B, which may be collectedfrom the sensors 20202A, 20204A corresponding to the other edge device20120A. Thus, in this embodiment, data from sensors 20202A, 20204A maybe stored in both edge storage 20122A and edge storage 20122B. Inembodiments, the redundant data 20222B, 20224B may be identical to thedetailed data 20212A, 20214A stored in another edge storage.Additionally or alternatively, the redundant data may include less data(e.g., a shorter history of data, fewer time-based samples of data,etc.) and/or higher grain data (e.g., summaries of certain data valuesbut not others). The aggregators 20140 may cause the edge devices 20120Aand 20120B to be in communication continuously, periodically, orotherwise to transfer and store redundant data.

In embodiments, reference tables 20216 may be stored in the edge storage20122 (e.g., reference tables 20216A in edge storage 20122A andreference tables 20216B in edge storage 20122B). Reference tables 20216may include various data and/or metadata describing the structure ofother data stored within the distributed database. For example,reference tables 20216 may indicate the structure (e.g., the columnvalues and data types) of data tables stored in other edge storage 20122connected to other edge devices 20120, such that a particular edgedevice (e.g., edge device 20120A) may be aware of the formats of otherdata tables stored in edge storage connected to other edge devices.Additionally or alternatively, the reference tables 20216 may indicatepermitted values for data tables stored elsewhere in the distributeddatabase. In embodiments, the reference tables stored in one edgestorage may be identical to the reference tables stored in another edgestorage, such that the same reference tables may be replicatedthroughout the distributed database, thus providing comprehensiveknowledge of the structure of the various data tables throughout thedistributed database. In embodiments, the reference data may bedifferent between edge nodes, thus creating result sets that aredifferent (but within tolerance levels managed by the dynamic ledger andaggregators).

In embodiments, query logs 20218 may be stored in the edge storage 20122(e.g., query logs 20218A in edge storage 20122A and query logs 20218B inedge storage 20122B). The query logs 20218 may contain a log of pastqueries received by the connected edge device and/or other edge devices.In embodiments, the query logs may be used to build predictive querymodels that may be used to predict which types of queries are mostfrequent, which types of data are most commonly queried, when particulartypes of data will be requested, and/or the like in order to improve theefficiency of the system, as described in more detail below.Additionally or alternatively, aggregators 20140 may continuously orperiodically review the query logs to optimize the distribution of datathroughout the distributed database. For example, an aggregator 20140may analyze a query log 20218 to detect repeated queries received at anedge device 20120 that the edge device 20120 was not able to execute(e.g., because sufficient data for responding to the query was notstored locally). Based on detecting the repeated queries, for example,the aggregator 20140 may cause the edge device 20120 to store data(e.g., dynamic ledger data 20220) for responding to the query in thefuture.

In embodiments, dynamic ledger data 20220 may be stored in the edgestorage 20122 (e.g., dynamic ledger data 20220A in edge storage 20122Aand dynamic ledger data 20220B in edge storage 20122B). The dynamicledger data may include any of the data stored on the dynamic ledger, asdiscussed in more detail below. In embodiments, aggregators 20140 may beresponsible to distributing various dynamic ledger data to various edgedevices in order to optimize performance of the network, allow edgedevices to quickly provide approximate responses to queries theyfrequently receive, allow edge devices to quickly provide approximateresponses to predicted future queries, instruct edge nodes to move data,and/or the like. In other words, the aggregators 20140 may continuallydistribute dynamic ledger data 20220 throughout the edge network, orinstruct edge nodes to do so, so that the data is likely to be where itis most needed. In embodiments, aggregators 20140 may generate and/ortransmit dynamic ledger data 20220 retrieved from the dynamic ledger tothe edge devices 20120 (e.g., as specified by one or more shardalgorithms). Additionally or alternatively, an aggregator 20140 mayinstruct one edge device 20120 to generate and/or transmit dynamicledger data to another edge device 20120 (e.g., as specified by one ormore sharing algorithms).

FIG. 167B illustrates details of example data stored in the dynamicledger 20150, which may be implemented and/or maintained by variousaggregators 20140 and/or edge devices/aggregators 20130. The aggregators20140 and/or edge devices/aggregators 20130 may be connected to variousedge devices 20120, each with their own edge storage 20122 andassociated sensors 20202 and/or other data sources 20204. Inembodiments, the edge devices/aggregators 20130 may further includeand/or communicate with sensors, data sources, and edge storage (e.g.,edge device/aggregator 20130A may communicate with sensors 20202C, datasources 20204C, and edge storage 20122C), thus acting as an edge device20120 as described above for FIG. 167A.

In embodiments, the dynamic ledger 20150 may include various data forresponding to queries and optimizing the functionality of thedistributed database. In embodiments, the dynamic ledger may beconfigured to allow the distributed database to leverage the massivestorage, processing power and memory of the edge devices to processqueries without having to transmit large quantities of data. The dynamicledger may allow for probabilistic views of the data to be stored in acentralized manner, allowing centralized queries without the need forlarge data streams. These probabilistic views of data may include aprobability distribution of the data and/or data outliers, both of whichmay be stored on a dynamic ledger. By combining communication andmanagement to handle these two kinds of data, the edge devices and/oraggregators can provide accurate query results without having totransmit fine grained data centrally (although fine-grained queries maybe executed at the edge node).

In embodiments, the dynamic ledger 20150 may contain “higher grain” datathan the data stored in edge storage, and thus may contain sensorsummary data 20252 and/or data source summary data 20254. For example,sensor summary data 20252 may include averages of sensor values byregion, by time, or by some other variable, maximums and/or minimums byregion, time, or some other variable, distribution data, and/or othersuch data that may be used to approximate or provide at least partialresponses to queries without requiring network requests to be sent to alarge number of edge devices. An example of using summary data torespond to query is provided in more detail below.

In embodiments, the dynamic ledger 20150 may include edge data locationdata 20256, which may indicate where various data may be found in edgestorage across the distributed network. Edge data location data 20256may indicate, for example, that data for a particular sensor/data sourceor set of sensors/data sources is stored at a particular edge device,that particular types of data are stored at particular sets of edgedevices, that data associated with particular locations is stored atparticular edge devices or sets of edge devices, and/or the like.Additionally or alternatively, the edge data location data 20256 mayinclude data used by various shard algorithms, which may be used toidentify a particular edge device that stores or should store (e.g., foran insert operation) a data value or set of data values.

In embodiments, the dynamic ledger 20150 may include edge device roledata 20258, which may indicate various roles that edge devices may takein the distributed database. In some embodiments, each of the edgedevices may take a uniform role. However, using uniform roles may createa situation in which each device must communicate with many otherdevices in the distributed database to obtain sufficient metadata forcluster operation. Additionally or alternatively, in some embodimentsvarious devices may take on various roles, such as in distributeddatabases with localized needs-based devices, where devices may performin one role out of two or more. For example, devices in a first role maybe responsible for collecting, managing, and/or maintaining one type ofdata, whereas devices in a second role may be similarly responsible fora second type of data. This role-based approach may bring the advantagethat metadata management may be isolated to only nodes in a particularrole. Thus, for example, a distributed database may use many differentkinds of edge devices based on the nature and type of operations beingperformed by the edge devices.

In embodiments, the dynamic ledger 20150 may include probabilitydistribution models 20260, which may be used to provide approximateanswers to queries or partial queries. For example, a probabilitydistribution models may indicate means/medians/modes (e.g., overall, fora particular region, for a particular time frame, etc.), standarddistributions, frequency distributions matrices, maximums, minimums,outlier values, etc. for various sensors, sensor types, regions, etc.The probability distribution models 20260 may enable (at leastapproximately) responding to a query that requests, for example, anaverage sensor reading for a region without requiring networkcommunications with most or all of the edge devices in the region. Inembodiments, the probability distribution models 20260 may beimplemented as, for example, trained neural networks or other machinelearning models that may be trained to predict values (e.g., averagesensor readings for a particular time of day) based on historical datastored in the distributed database. The probability distribution models20260 may be determined/trained/etc. by the edge devices 20120, edgedevice/aggregators 20130, and/or aggregators 20140.

In embodiments, the dynamic ledger 20150 may include query predictionmodels 20262, which may be used to predict future queries for anupcoming time period, determine the most common future query, etc. Thequery prediction models 20262 may be implemented as, for example,trained neural networks or other machine learning models that may betrained to predict future queries based on historical query data (whichmay be stored as query logs 20218 in edge storage 20122 or query logs20266 stored in the dynamic ledger 20150). The query prediction models20262 may be trained by the edge devices 20120, edge device/aggregators20130, and/or aggregators 20140. The query prediction models may be usedto prepare summary data and/or update distribution models in advance sothat the dynamic ledger 20150 stores the most relevant data foroptimizing the operation of the distributed database, as described inmore detail below with respect to FIG. 169D.

In embodiments, the dynamic ledger 20150 may include reference tables20264, which may be identical or distinct from the reference tables20216 stored in the edge storage 20122. Additionally or alternatively,the dynamic ledger 20150 may include query logs 20266, which may includedata taken from various query logs stored in edge storage (e.g., a firstset of queries from query logs 20218A, a second set of queries fromquery logs 20218B, etc.).

In embodiments, the dynamic ledger 20150 may include pending datarequests 20268, which may include pending queries or other data requeststhat may be monitored by edge devices in order to respond with requesteddata. In embodiments, although the summary data 20252, 20254 and/orprobability distribution models 20260 may enable responses (e.g.,approximate responses) to some queries, for other queries data may needto be retrieved from edge devices. Additionally or alternatively,summary data 20252, 20254 and/or probability distribution models 20260may need to be continually updated/retrained in order to incorporate thelatest data, in order to respond to future predicted queries, and/or thelike. Accordingly, pending data requests 20268 may be stored in thedynamic ledger 20150 and monitored by the various edge devices 20120. Apending data request 20268, for example, may include a formatted query(e.g., the identical query received from a query device 20110, a portionof a query, etc.). Edge devices 20120 may monitor the pending datarequests 20268 and (e.g., when resources are available), upload data(e.g., data stored in edge storage) to an aggregator 20140 forprocessing and/or process the data themselves in order to update summarydata, retrain a probability distribution model, respond to a query,and/or the like. In embodiments, aggregators 20140 may maintain a listof pending data request 20268 in priority order such that edge devices20120 respond to the most important pending data requests first whenresources are available. Additionally or alternatively, the pending datarequests may be ordered in chronological order such that the edgedevices respond to the oldest requests first (e.g., first in first out).

In embodiments, device software (e.g., configured modules running onedge devices and/or aggregators) may have automatic and/or configurableedge-sharding (also referred to herein as “partitioning”) built in. Someimplementations of the modules may be targeted more at transactionalworkloads, and some at analytical workloads, many of the products maycombine both kinds of workloads into one. Database modules mayaccomplish this combination by the use of a multi-layered architectureusing microservice components or modules.

FIG. 168A illustrates example modular components of an example edgedevice 20120A. In embodiments, each edge device may have the illustratedmodules and/or other modules, which may be implemented as microservicesrunning on the edge device 20120A. Additionally or alternatively,different edge devices 20120 can have different modules. For example,aggregators 20140 may cause different modules to be distributed todifferent edge devices 20120 according to roles of different edgedevices, data available to different edge devices, to dynamically handledifferent queries or other loads at different parts of a network, and/orthe like. Accordingly, the illustrated modules are example modules thatmay not be replicated and/or used on every edge device, may be switchedon or off dynamically, and the like.

In embodiments, an edge device 20120 includes an API module 20302 forreceiving queries (e.g., from query devices 20110) and routing thereceived queries to other modules for processing, receiving instructionsfrom aggregators and routing the instructions to other modules forprocessing, and/or otherwise interfacing between the other modules ofthe edge device 20120 and/or other devices. In embodiments, the APImodule allows a user to interact with tables and data stored inside thedistributed database as the queries are running against a single serverrelational database. Users may use the API to insert, update/delete,perform join operations, or select data from tables (e.g., for a webapplication). As shown in the figure, the API module 20302 may be incommunication with query devices 20110 (e.g., for receiving queries andtransmitting query responses), other edge devices 20120B-N (e.g., toshare redundant data as instructed by an aggregator 20140), and/oraggregators 20140 (e.g., to receive instructions for operation, fortransmission of data, etc.).

In embodiments, an edge device 20120 includes a modelling module 20304for building probability distribution models 20260, generating estimatesusing the models 20260, causing storage of the models on a dynamicledger 20150, calculating statistical confidence, and/or the like. Forexample, an aggregator 20140 may instruct an edge device 20120 to buildand/or maintain (e.g., keep updated) a probability distribution modelfor data stored in edge storage 20122 maintained by the edge device, andto cause the probability distribution model to be kept updated on thedynamic ledger 20150. Accordingly, the edge device 20120, using themodelling module 20304, may continuously develop the model (e.g.,continuously update various statistical measurements such as a mean,standard deviation, etc., and/or continuously retrain a neural networkor other machine learning model) based on data that may be kept in theedge storage 20122. The edge device 20120 may further cause the updatedmodel to be stored on a dynamic ledger if instructed by the aggregator20140 (e.g., by transmitting the updated model to the aggregator 20140,which may cause it to be stored on the dynamic ledger). Furthermore, inembodiments, the modelling module 20304 may use one or more of theprobability distribution models 20260 to respond to a query received bythe API module 20302. For example, if a query requesting a sum ofvalues, an average of a particular value, a count of a particular value,etc. is received, and the edge device 20120A has access to theprobability distribution model 20260 (whether generated by that edgedevice 20120A or some other device), the edge device 20120A may use theprobability distribution model 20260 to provide an approximate answer tothe query.

In embodiments, the modelling module 20304 may be used to continuouslyupdate and/or retrain query prediction models 20262. For example, themodelling module 20304 may continuously retrain a neural network orother machine learning model using data taken from query logs, such thatthe query prediction models 20262 are kept up to date and the edgedevice 20120A can more accurately predict what types of queries it willreceive. In embodiments, the edge device 20120A may cause the updatedquery prediction models 20262 to be stored on the dynamic ledger 20150.

In embodiments, an edge device 20120 includes a dynamic ledger module20306 for reading data from the dynamic ledger 20150 and/or writing datato the dynamic ledger 20150, as well as monitoring the dynamic ledger20150 (e.g., for pending data requests 20268 that may require the edgedevice 20120 to take action). In some embodiments, the dynamic ledgermodule 20306 may have functionality for reading data from the dynamicledger 20150 but not writing to the dynamic ledger 20150 (e.g., if theaggregators are responsible for writing data to the dynamic ledger20150). In these embodiments, the dynamic ledger module 20306 maytransmit data to the aggregator 20140 in order to write data to thedynamic ledger 20150. In embodiments, the dynamic ledger module 20306may cause the updated modelling data to the stored on the dynamic ledger20150, as discussed above.

In embodiments, an edge device 20120 includes a query execution module20308 for determining whether the edge device 20120 has sufficient datato respond to a query, for creating query plans, for executing queriesor partial queries against the edge storage, for causing the modellingmodule 20304 to generate approximate responses to queries or partialqueries, and/or the like. In embodiments, the query execution module20308 may be configured to deliver an efficient query plan with minimalresource consumption and fast response time. For example, in order toavoid bottlenecks on a single node (e.g., edge device 20120A), queryexecution may be spread across nodes in the edge network. Furthermore,the query execution module 20308 may use the ability for localized edgesystems to have seamless access to their required data, even in periodsof sporadic network connectivity. In embodiments, the query executionmodule 20308 may reject a query when sufficient data for answering thequery is not obtainable by the edge device 20120. Additionally oralternatively, in some embodiments the query execution module 20308 maydetermine that at least part of the query may be satisfied using data inedge storage 20122, that at least part of the query may be satisfiedusing models on a dynamic ledger 20150, and/or the like, may generate aquery plan for responding to the query, and/or may execute the queryplan. For example, if a query comprises an expression with two types ofdata, the query execution module 20308 may generate a query plan forexecuting the query using any method of obtaining the two types of dataor approximations thereof. Continuing the example, if the querycomprises an expression like SUM (value1)/AVERAGE(value2), the queryexecution module 20308 might determine that the SUM(value1) querycomponent may be obtained from edge storage 20122, and theAVERAGE(value2) query component may be obtained using a probabilitydistribution model 20260. The query execution module 20308 may thusgenerate a query plan for obtaining the necessary data andapproximations and estimating the value of the expression. Inembodiments, the query results may include a confidence factor relatingto the accuracy of the result dataset.

In embodiments, an edge device 20120 includes an edge storage module20310 for interfacing with edge storage 20122 in order to cause inserts,updates, deletes, joins, selects, and/or other query languageoperations/statements. In embodiments, the edge storage module 20310 mayhandle the automatic sharding of data across nodes (e.g., edge devices20120) in a particular edge cluster. Edge sharding may optimize queryperformance for both edge aggregate queries and filtered queries withlogic predicates. The distributed database system thus allows scaling byadding more edge devices, increasing capacity and performance linearly.Edge storage architecture allows for scaling out the edge serviceshorizontally based on demand. Various different architectures may beused to achieve optimized execution and user experience in differentimplementations with different target workloads, as described in moredetail below.

The edge storage module 20310 may implement, for example, one or moreSQL functions with additional functionality provided by DML and/or DDLextensions in order to cause the edge device 20120A to implement EDQL.In embodiments, the edge storage module 20310 may use one or morereference tables to allow the edge device 20120 to operate even whencertain data tables are not stored in the edge storage 20122, asdiscussed elsewhere herein.

FIG. 168B illustrates modular components of an example aggregator20140A. In embodiments, each aggregator may have the illustrated modulesand/or other modules, which may be implemented as microservices runningon the aggregator 20140A. Additionally or alternatively, differentaggregators 20140 can have different modules. For example, variousaggregators 20140 may coordinate to cause different modules to bedistributed to different aggregators 20140 in order to dynamicallyhandle different loads at different parts of a network. Accordingly, theillustrated modules are example modules that may not be replicatedand/or used on every aggregator 20140. Additionally or alternatively,even if the modules are replicated across every aggregator, they may beswitched on or off dynamically (e.g., such that different aggregatorsmay perform different roles) and the like.

In embodiments, an aggregator 20140 includes an API module 20352 forreceiving queries (e.g., from query devices 20110) and routing thereceived queries to other modules for processing, transmittinginstructions to edge devices 20120, receiving data (e.g., data forstorage on the dynamic ledger 20150) from edge devices 20120 and routingthe data to other modules for processing, and/or otherwise interfacingbetween the other modules of the aggregator 20140 and/or other devices.As shown in the figure, the API module 20352 may be in communicationwith query devices 20110 (e.g., for receiving queries and transmittingquery responses) and edge devices 20120.

In embodiments, an aggregator 20140 includes an edge data managementmodule 20354 for finding the location of edge data (e.g., which edgedevice 20120 have data and/or which do not), for determining which edgedevices should store data (e.g., using shard algorithms, edge datalocation data 20256, or other data to determine where to store data),for determining where redundant data should be stored (e.g., based onquery prediction models 20262), and for transmitting data and/orinstructions that cause data to be stored at the appropriate edgedevices in order to optimize the system, enable edge devices toefficiently respond to queries, and/or the like. In embodiments, theedge data management module 20354 may use shard algorithms that indicatea particular edge device that does (or should) store data.

In embodiments, the edge storage module 20310 may handle the automaticsharding of data across nodes (e.g., edge devices 20120) in a particularedge cluster. Edge sharding may optimize query performance for both edgeaggregate queries and filtered queries with logic predicates. Thedistributed database system thus allows scaling by adding more edgedevices, increasing capacity and performance linearly. Edge storagearchitecture allows for scaling out the edge services horizontally basedon demand. Various different architectures may be used to achieveoptimized execution and user experience in different implementationswith different target workloads, as described in more detail below.

In embodiments, the edge data management module 20354 may use varioustypes of shard algorithms. For example, the edge data management module20354 may use a distributed logical shard algorithm (e.g. an algorithmthat is distributed within an edge cluster and is based on a set oflogical rules, such as column values), where a particular aggregator20140 may have a certain set of edge data location data 20256 thatprovides the logical rules for edges “nearby” (e.g., in a same localnetwork or other logical portion of the network) the aggregator in thenetwork. Additionally or alternatively, the edge data management module20354 may use a local neural network (e.g., where the shard algorithm islocal to a specific cluster or is based on a neural network).Additionally or alternatively, the edge data management module 20354 mayuse a local genetic network (e.g., a shard algorithm that is local to aspecific cluster and is based on a genetic algorithm network).

In embodiments, the edge data management module 20354 may use queryprediction models 20262 and/or query logs stored in edge storage and/orthe dynamic ledger to detect whether queries are being efficientlyhandled (e.g., whether edge devices have sufficient data to respond toqueries, whether responses use sufficient data to provide anapproximation of sufficient accuracy, etc.). In embodiments, the edgedata management module 20354 may thus continuously (e.g., constantly,periodically) review the past and predicted performance of the edgedevices and cause modifications to the distribution of edge datathroughout the system in order to improve operation of the distributeddatabase. For example, the edge data management module 20354 maydiscover frequent queries to a particular edge device 20120A where theparticular edge device 20120A does not have sufficient data to respondto the query and may accordingly determine that data stored at one ormore other edge devices 20120B-N should be transmitted to the edgedevice 20120A in order to provide better responses to future queries.The edge data management module 20354 may use past query logs and/orquery prediction models 20262 to determine that data needs to beredistributed. For example, the edge data management module 20354 mayuse query prediction models to predict a large volume of incomingqueries to a particular edge device 20120A and may cause other edgedevices 20120B-N to develop probability distribution models 20260 fordata needed to respond to the predicted queries, store the probabilitydistribution models 20260 to the dynamic ledger 20150, and/or transmitthe probability distribution models 20260 to the edge device 20120A inadvance of the predicted queries.

In embodiments, the edge data management module 20354 may determine thatnew data (e.g., a new table, a new row for an existing table, etc.)should be stored at a particular edge device. The edge data managementmodule 20354 may use shard algorithms on the new data to determine whichedge device(s) should store the new data and may transmit the new datato the corresponding edge devices accordingly.

In embodiments, an aggregator 20140 includes a query planning module20356 for creating query plans, distributing queries and/or partialqueries to edge devices 20120, executing queries and/or partial queriesusing dynamic ledger data, and/or the like. In embodiments, the queryplanning module 20356 may determine that at least part of the query maybe satisfied using data stored by various edge devices 20120A-N, maygenerate a query plan for distributing partial queries to the variousedge devices 20120A-N, and may execute the query plan. For example, if aquery comprises an expression including a range of data distributedacross various devices, the query planning module 20356 may generate aquery plan for transmitting partial queries to at least some of thevarious devices (e.g., all, a representative sample, etc. based on thequery). Additionally or alternatively, the query planning module 20356may generate a query plan for using dynamic ledger data to generateapproximate data matching at least part of the query. Thus, the queryplanning module 20356 may generate query plans that can be executedlocally and/or may involve transmitting data requests to otheraggregators 20140B-N, edge devices, etc.

In embodiments, an aggregator 20140 includes a modelling module 20358for generating estimates using probability distribution models 20260,causing storage of the models 20260 on a dynamic ledger 20150 (e.g.,when the models are generated by edge devices 20120 and received by theaggregator 20140), calculating statistical confidence, and/or the like.For example, an aggregator 20140 may receive a probability distributionmodel for data stored in edge storage 20122 and cause the probabilitydistribution model 20260 to be stored on the dynamic ledger 20150.Furthermore, in embodiments, the modelling module 20358 may use one ormore of the probability distribution models 20260 to respond to a queryreceived by the API module 20352. For example, if a query requesting asum of values, an average of a particular value, a count of a particularvalue, etc. is received, and the aggregator 20140 has access to theprobability distribution model 20260, the aggregator 20140 may use theprobability distribution model 20260 to provide an approximate answer tothe query.

In embodiments, an aggregator 20140 includes a dynamic ledger module20360 for reading data from the dynamic ledger 20150 and/or writing datato the dynamic ledger 20150. In some embodiments, the dynamic ledgermodule 20360 may be responsible for writing data that is received fromedge devices to the dynamic ledger 20150. In some embodiments, thedynamic ledger contains instructions for the edge devices to move oraggregate data. In some embodiments (e.g., embodiments where the dynamicledger is a blockchain), the dynamic ledger module 20360 may implementconsensus mechanisms and otherwise cause the aggregator to act as ablockchain node (e.g., by mining new blocks, etc.).

Different embodiments of the distributed database system may usedifferent database architectures depending, for example, on targetworkload. A first example system architecture is a shared distributedledger architecture. In a shared distributed ledger architecture,compute nodes (e.g., edge devices 20120 and/or aggregators 20140) mayaccess a common memory address space via a high-speed network. In thisarchitecture, the dynamic ledger may be shared between nodes and may beused by query modules (e.g., the query planning module 20356 and/or thequery execution module 20308) to decide on how and where to allocatequery resources. This implementation may hold a shared dynamic ledger.

A second example system architecture is a shared storage architecture.In a shared storage architecture, compute nodes (e.g., edge devices20120 and/or aggregators 20140) may be independent of durable storage.Compute nodes may have local memory and a buffer pool for ephemeraldata, which may cause a penalty for not having data locality. In theseembodiments, updates may require messaging between compute nodes (e.g.,as determined by aggregators 20140) to notify each node/device of achanged state. Rather than distribute storage, the shared storage modelmay construct a centralized storage model and hold all data from allnodes in this shared storage. The edge devices may then provide the CPUand processing to execute queries against this shared storage. In manyimplementations, there may be an amount of shared storage (e.g., boththe shared distributed ledger and the shared storage architecture may becombined into a single system for different data).

A third example system architecture is a shared nothing architecture. Ina shared nothing architecture, each node (e.g., edge devices 20120and/or aggregators 20140) may have its own local CPUs, memory, and localstorage. This architecture may offer the best performance and efficiencyin some cases due to data locality, thus moving the least amount of dataacross the network. In this architecture, the implementation is highlydistributed, and nodes may not share information regarding their data.In the shared service mode, queries may be placed in a bulletin boardpattern query request area (e.g., as pending data requests 20268) andone or more edge agents (e.g., edge devices 20120 and/or aggregators20140) can either resolve or partially resolve a query for a trulyshared nothing implementation.

FIG. 169A illustrates an example method 20400 for receiving andresponding to queries according to embodiments described herein. Thesteps shown in FIG. 169A may be executed by any of an edge device 20120,an aggregator 20140, and/or an edge/device aggregator 20130. Forpurposes of illustration, the steps will be described as being executedby an edge/device aggregator 20130.

At 20402, an edge device/aggregator 20130 may receive a query from aquery device 20110. The query may request data stored in the distributeddatabase. In some embodiments (and/or depending on the query), the edgedevice/aggregator 20130 may then execute a partial query against datastored in edge storage that is connected to the edge device/aggregator20130. For example, if the query requests a most recent sensor readingfrom all devices within a region, and the edge device/aggregator 20130is connected to edge storage with a most recent sensor reading for someof the devices within the region, the edge device/aggregator 20130 mayexecute a partial query by retrieving the matching most recent sensorreadings that are stored in connected edge storage. In some embodiments(and/or depending on the query and/or retrieved data), the partial queryexecuted at 20404 may yield enough data to respond to the query receivedat 20402 (e.g., when query requests an average and the partial queryyields a statistically significant amount of data for responding to thequery), as described in more detail below with respect to FIG. 169B.However, in the example of FIG. 169A, the method may proceed to step20406.

At 20406, the edge device/aggregator 20130 may cause storage of thequery received at 20402 on the dynamic ledger 20150. The query may bestored on the dynamic ledger 20150 as a pending data request 20268 sothat other edge devices can retrieve and respond to the query (e.g.,when sufficient network resources are available). For example, edgedevices 20120, aggregators 20140, and/or edge device/aggregators 20130may continually monitor queries posted to the dynamic ledger 20150 usinga dynamic ledger module 20306 and/or a dynamic ledger module 20360, asdescribed above. When a query is detected as a pending data request20268 on the dynamic ledger 20150 by one or more edge devices and/oraggregators, the edge devices and/or aggregators may generate summarydata 20252, 20254 and/or other data for generating probabilitydistribution models 20260 (e.g., from data stored on connected edgestorage and/or edge storage associated with a connected edge device) andcause the summary data and/or probability distribution models to beuploaded to the dynamic ledger 20150. The process of monitoring thedynamic ledger 20150 and responding to pending data requests 20268 isdescribed in more detail below with respect to FIG. 169C.

At 20408, the edge device/aggregator 20130 may wait until summarydata/other data for generating probability distribution models 20260 isuploaded to the dynamic ledger 20150. For example, the edgedevice/aggregator 20130 may wait until a certain number or percentage ofthe edge devices and/or aggregators with matching data stored in edgestorage have responded. In embodiments, the edge device/aggregator 20130may stop waiting after a predetermined amount of time (e.g., a timeoutvalue) if sufficient data has not yet been received, and then eitherproceed with the method (e.g., if a reasonably accurate respond to thequery can be provided using the uploaded summary data) and/or indicatethat the query cannot be satisfied (e.g., if sufficient data has notbeen received to approximate an answer).

At 20410, the edge device/aggregator 20130 may generate a probabilitydistribution model 20260 from the summary data 20252, 20254 stored onthe dynamic ledger 20150. In some embodiments, the device that generatesthe probability distribution model 20260 at 20410 and the device thatreceives the query at 20402 are the same device. Additionally oralternatively, the devices may be different (e.g., an edge device 20120may receive the query at 20402 and an aggregator 20140 may generate theprobability distribution model 20260 at 20410). The probabilitydistribution model 20260 may be generated based on the query received at20402. For example, if the query requests an average of a specific typeof sensor reading, the probability distribution model 20260 mayrepresent the distribution of sensor readings for that specified type ofsensor. Thus, the probability distribution model 20260 may enable anapproximate respond to the query received at 20402 and/or an approximaterespond to a future query requesting the same or similar data. Inembodiments, the edge device/aggregator 20130 may cause the probabilitydistribution model 20260 to be stored on the dynamic ledger 20150 sothat it may be used to respond to future queries.

At 20412, the edge device/aggregator 20130 may generate a response (orapproximate response) to the query received at 20402 using one or moreof the probability distribution models 20260 (e.g., as generated at20410) and/or the partial query results (e.g., responsive to the partialquery at 20404). In embodiments, the edge device/aggregator 20130 mayuse the partial query results together with the probability distributionmodel 20260 to provide a more accurate approximation (e.g., depending onthe query). Then, at 20414, the generated query may be transmitted tothe query device 20110. In some embodiments (not shown), if the edgedevice/aggregator 20130 is unable to provide a reasonably accurateapproximation in response to the query received at 20402 (e.g., becausea timeout is reached before enough summary data is uploaded to thedynamic ledger 20150), the edge device/aggregator 20130 may transmit aresponse indicating that the query cannot be fulfilled.

At 20416, a second query may be received from the same query device20110 or a different query device 20110. In embodiments, the queryreceived at 20416 may be the same as the query received at 20402, and/ormay request overlapping and/or similar data as the query received at20402. In these and similar embodiments, at 20418 the edgedevice/aggregator 20130 may be able to respond to the second query usingthe previously generated probability distribution model 20260 and/orpartial query results. Additionally or alternatively, the edgedevice/aggregator 20130 may also run a second partial query againstlocal edge storage (e.g., it may repeat step 20404 in order to retrieveupdated data in response to the second query) and may use the secondpartial query results to generate a better approximate second queryresponse.

At 20420, the second query response may be transmitted to the querydevice 20110. Thus, as shown by the example method 20400, thedistributed database may improve its ability to provide approximateresponses to queries over time as queries are received. Accordingly, insome embodiments, a response to a second query may be generated andtransmitted more quickly and/or more accurately than a response to afirst query, which may be the same or similar to the second query.

In embodiments (not shown in FIG. 169A), the edge device/aggregator20130 may periodically update and/or refine particular probabilitydistribution models 20260. In some embodiments, the edgedevice/aggregator 20130 may thus repeat steps 20406-20410 on a periodicbasis in order to update the probability distribution models 20260. Forexample, the edge device/aggregator 20130 may more frequently updateprobability distribution models 20260 that are used more often and mayless frequently update probability distribution models 20260 that areused less often. Additionally or alternatively, the edgedevice/aggregator 20130 may more frequently less accurate probabilitydistribution models 20260 (e.g., models 20260 that were generated and/ortrained using less data) in order to improve the probabilitydistribution models 20260 most in need of improvement. Accordingly,because probability distribution models 20260 may be refined over time,approximate responses to queries may become more accurate over time. Inembodiments, the aggregator provides instructions onto the dynamicledger for edge nodes to build and maintain probability distributionmodels.

FIG. 169B illustrates an example method 20430 for receiving andresponding to a query according to embodiments described herein. Thesteps shown in FIG. 169B may be executed by any of an edge device 20120,an aggregator 20140, and/or an edge/device aggregator 20130. Forpurposes of illustration, the steps will be described as being executedby an edge/device aggregator 20130.

At 20432, an edge device/aggregator 20130 may receive a query from aquery device 20110. The query may request data stored in the distributeddatabase. In the illustrated embodiment (and/or depending on the query),at 20434 the edge device/aggregator 20130 may then execute a partialquery against data stored in edge storage that is connected to the edgedevice/aggregator 20130. For example, if the query requests an averagepower consumption for a particular type of device within a region, andthe edge device/aggregator 20130 is connected to edge storage with powerconsumption data for some of the devices within the region, the edgedevice/aggregator 20130 may execute a partial query by retrieving thematching power consumption data that is stored in connected edgestorage.

In some embodiments (and/or depending on the query and/or retrieveddata), the partial query executed at 20434 may yield enough data togenerate a model for responding to the query received at 20432. Forexample, if the query requests average power consumption data for aspecific type of devices, and the local edge storage contains enoughdata to provide a statistically accurate estimate of the average powerconsumption, the edge device/aggregator 20130 may be able to provide aresponse to the query without obtaining data from other devices. Thus,for example, at 20436 the edge device/aggregator 20130 may proceed tobuild a probability distribution model 20260 based on a partial queryresponse received in response to the partial query of 20434. The edgedevice/aggregator 20130 may use any method of generating the probabilitydistribution model 20260 as described herein.

At 20438, the edge device/aggregator 20130 may ensure that anapproximate answer to the query can be provided to a certain statisticalconfidence. For example, based on the sample size of the sample datareceived in response to the partial query, a confidence intervalthreshold, and/or a standard deviation of the sample data, the edgedevice/aggregator 20130 may determine that the statistical confidence ofthe model is sufficient.

At 20440B, when the statistical confidence is sufficient, the edgedevice/aggregator 20130 may generate a query response based on the modelgenerated at 20436 and/or based on the partial query response. Then, at20442B, the generated query response may be transmitted to the querydevice.

By contrast, if the statistical confidence is not sufficient, the edgedevice/aggregator 20130 may proceed by causing storage of the query onthe dynamic ledger, then waiting for summary data to be uploaded to thedynamic ledger as described above for steps 20406-20408. Then, at20442A, the edge device/aggregator 20130 may generate a new model basedon the partial query response and/or the summary data uploaded to thedynamic ledger (e.g., as described above for step 20410). Next, at20440B, the edge device/aggregator 20130 may use the new model togenerate a query response and, at 20442B, transmit the query response tothe query device. In embodiments, the query results may be returned asfailed as the EDQL module is not enabled to handle this data.

Although the method 20400 of FIG. 169A and the method 20430 of FIG. 169Bare illustrated as separate example methods, in embodiments, the methods20400 and 20430 may be implemented by the same module and/or devicesdepending on the query received, the data (if any) obtained from edgestorage, and/or the like. Accordingly, the methods 20400 and 20430should be understood as different example flows that may be implementedby the same device in different conditions as appropriate.

FIG. 169C illustrates an example method 20450 for monitoring andresponding to a pending data request 20268 stored on the dynamic ledger20150 according to embodiments described herein. The steps shown in FIG.169C may be executed by any of an edge device 20120, an aggregator20140, and/or an edge/device aggregator 20130. For purposes ofillustration, the steps will be described as being executed by anedge/device aggregator 20130.

At 20452, the edge device/aggregator 20130 may continually collectdetailed data 20212, 20214 from sensors 20202 and/or other data sources20204, which may be part of and/or connected to the edgedevice/aggregator 20130 as discussed herein. For example, the edgedevice/aggregator 20130 may continuously record the detailed data,process it (e.g., format it, calculate data based on the data), store itin edge storage, and otherwise maintain the detailed data in edgestorage. In embodiments, the edge device/aggregator 20130 may analyzethe detailed data 20212, 20214 to generate additional detailed data20212, 20214 and store the additional detailed data 20212, 20214 in edgestorage. For example, the edge device/aggregator 20130 may collect andstore detailed electrical current data and may use the detailedelectrical current data to calculate and store power consumption data.As another example, the edge device/aggregator 20130 may collect andstore detailed video data of a production line and may use the detailedvideo data to calculate and store data indicating a count of itemsoutput by the production line. Thus, the detailed data stored at 20452may include data received from sensors 20202 and/or other data sources20204 as well as data derived therefrom.

At 20454, the edge device/aggregator 20130 may determine (e.g., asindicated by a dynamic ledger module 20306, 20360) that a query (orother pending data request 20268) has been posted to the dynamic ledger20150. For example, the edge device/aggregator 20130 may continuouslymonitor the pending data request 20268 stored on the dynamic ledger20150 in order to determine if there exists a pending data request 20268that can be at least partially responded to by the edgedevice/aggregator 20130. Additionally or alternatively, the edgedevice/aggregator 20130 may periodically check the pending data request20268 (e.g., at regular intervals, during downtime when a network and/orprocessing load is below a threshold, etc.). Additionally oralternatively, aggregators 20140 may monitor the pending data requests20268 and instruct certain edge devices 20120 to perform further actions(e.g., to execute steps 20456-20460) based on the pending data requests20268.

At 20456, the edge device/aggregator 20130 may execute a partial queryagainst local edge storage based on the query and/or other pending datarequest 20268 stored on the dynamic ledger 20150. For example, the edgedevice/aggregator 20130 may take the query as-is from the pending datarequests and run it against the detailed data stored in connected edgestorage. Then, at 20458, the edge device/aggregator 20130 may generatesummary data (e.g., higher grain data) for storage on the dynamic ledger20150 based on the pending data request. For example, the edgedevice/aggregator 20130 may calculate averages and/or other statisticalmeasurements of data received in response the partial query. At 20460,the edge device/aggregator 20130 may cause storage of the generatedsummary data on the dynamic ledger 20150. Additionally or alternatively,the edge device/aggregator 20130 may develop and/or train one or moreprobability distribution models based on the summary data and may causestorage of the one or more probability distribution models on thedynamic ledger 20150.

FIG. 169D illustrates an example method 20470 for predicting futurequeries and uploading data for responding to the future queries to thedynamic ledger 20150 according to embodiments described herein. Thesteps shown in FIG. 169D may be executed by any of an edge device 20120,an aggregator 20140, and/or an edge/device aggregator 20130. Forpurposes of illustration, the steps will be described as being executedby an edge/device aggregator 20130.

In embodiments, the method 20470 may be executed by various devicesduring downtime (e.g., reduced network and/or processing load) for thedistributed database. Additionally or alternatively, devices (e.g.,aggregators 20140) may continuously execute the method 20470 tocontinually anticipate future queries and prepare the distributeddatabase to handle the future queries.

At 20472, the edge device/aggregator 20130 may continually collectdetailed data 20212, 20214 from sensors 20202 and/or other data sources20204, which may be part of and/or connected to the edgedevice/aggregator 20130 as discussed herein. For example, the edgedevice/aggregator 20130 may continuously record the detailed data,process it (e.g., format it, calculate data based on the data), store itin edge storage, and otherwise maintain the detailed data in edgestorage. In embodiments, the edge device/aggregator 20130 may analyzethe detailed data 20212, 20214 to generate additional detailed data20212, 20214 and store the additional detailed data 20212, 20214 in edgestorage.

At 20474, the edge device/aggregator 20130 may execute a queryprediction model 20262 stored on the dynamic ledger 20150 in order topredict one or more future queries that may be received. In embodiments,the device executing step 20474 may predict future queries that may bereceived by that same device (e.g., using a query prediction model 20262trained on queries received by that same device and/or by similardevices). Additionally or alternatively, the device executing step 20474may predict future queries that may be received by a different device.For example, an aggregator 20140 executing step 20474 may predictqueries that may be received by one or more edge devices 20120 incommunication with the aggregator 20140.

At 20476, the edge device/aggregator 20130 may execute the predictedfuture query against local edge storage. In embodiments, the edgedevice/aggregator 20130 may repeatedly execute the predicted futurequery against local edge storage. For example, if a predicted futurequery is for an average of recent sensor data, the edgedevice/aggregator 20130 may continue to execute the predicted futurequery against local edge storage (e.g., in order to maintain anup-to-date model based on the latest data) until the query predictionmodel no longer predicts the future query.

At 20478, the edge device/aggregator 20130 may generate summary data(e.g., higher grain data) for storage on the dynamic ledger 20150 basedon the predicted future query. For example, the edge device/aggregator20130 may calculate averages and/or other statistical measurements ofdata received in response to the predicted future query. At 20480, theedge device/aggregator 20130 may cause storage of the generated summarydata on the dynamic ledger 20150. Additionally or alternatively, theedge device/aggregator 20130 may develop and/or train one or moreprobability distribution models based on the summary data and may causestorage of the one or more probability distribution models on thedynamic ledger 20150.

Although the method 20450 of FIG. 169C and the method 20470 of FIG. 169Dare illustrated as separate example methods, in embodiments, the methods20450 and 20470 may be implemented by the same module and/or devices.For example, aggregators 20140 may continually monitor pending datarequest as well as generate predicted future queries in order tocontinually generate models, prepare summary data, etc. for respondingto current or predicted future queries. Accordingly, the methods 20450and 20470 should be understood as different example flows that may beimplemented by the same device in order to continuously optimize thedistributed database.

In embodiments, in addition to or as an alternative to the methods 20450and/or 20470, aggregators 20140 (for example) may continuously causedetailed data to be moved from one edge device 20120 to be stored asredundant data at another edge device 20120 (e.g., as shown at FIG.167A). For example, the aggregators 20140 may detect (e.g., from querylogs that one edge device 20120 continuously receives requests for datathat is stored at another edge device 20120, and accordingly may causethe requested data to be stored redundantly at the edge device 20120receiving the queries. Thus, in addition to optimizing the distributeddatabase by preparing summary data and/or models to provide approximateresponses, the aggregators 20140 may further optimize the distributeddatabase by moving data from edge device 20120 to edge device 20120 inorder to allow edge devices 20120 to better handle queries.

FIGS. 170A-B illustrate example data flows 20500, 20550 for generatingdata structures to be stored in the distributed database using a querylanguage (e.g., EDQL) as described herein. In the illustrated examples,queries 20502 are formatted in an SQL format with custom DDL extensions(e.g., EDQL format). However, it should be understood that, in someembodiments, the queries may be formatted in other query languages. Theexample data flows 20500, 20550 illustrate specific DDL concepts thatmay be used to provide a database schema in the context of the edgedistributed system as described herein.

In embodiments, some or all of the edge distributed tables of thedistributed database may be associated with one or more shard algorithm(also referred to herein as a “shard lookup algorithm”). The shardalgorithm may function like a normal table index and may contain anynumber of columns. The shard algorithm may be used to determine whichpartition (or partitions) a given row belongs to (e.g., which edgestorage 20122 should store the given row of a distributed table).

When a query contains an INSERT, a CREATE, or a similar statement (e.g.,according to an SQL standard), the edge data management module 20354 ofthe aggregator 20140 may compute an output value based on the values inthe column or columns using a shard algorithm, may perform a splitteralgorithm operation to get an edge partition index (e.g., an identifierof a particular edge device 20120 and/or edge storage 20122), and maydirect the query to the appropriate partition(s) on the edge device(s)(e.g., to a given edge storage 20122). In embodiments, any two rows withthe same shard algorithm value may be guaranteed to be on the samepartitions due to the operation of the shard algorithm.

In embodiments, query optimizers (e.g., a query planning module 20356 ofthe aggregator 20140 and/or a query execution module 20308 of an edgedevice 20120) may leverage shard algorithms to determine how a queryshould be executed. For example, queries that fully match one or moreshard algorithm parameters may be routed directly to a single partitionon a single edge device (e.g., as shown in FIG. 170A). Other queries(e.g., group-by queries) where the set of keys do not overlap betweenpartitions can be executed in parallel on the distributed edges. In someembodiments (e.g., depending on the query), the results may be streamedback without any additional processing on the edge data managementmodule 20354.

FIG. 170A illustrates an example in which a query 20502 for creating anew data table is provided to an aggregator 20140, which causes an edgedevice 20120A to store the data table in edge storage 20122A. Theexample query 20502 of FIG. 170A uses a specified shard algorithm(labelled “local_neural” in the example query) and a designated primarykey (labelled “event_id” in the example query) as an argument to theshard algorithm. In the illustrated example, the specified shardalgorithm argument causes the distributed table to be stored on a singleedge partition (e.g., because the indicated shard algorithm is a neuralnetwork that, in the example case, outputs to a single edge storagepartition 20122A). Additionally or alternatively, another example querythat creates a table with a primary key and no explicit shard algorithmmay operate in a similar manner (e.g., the primary key may be used asthe shard algorithm parameter by default). In embodiments, using theprimary key as a shard algorithm parameter may help avoid data skewbecause it may cause an even distribution of data.

FIG. 170B illustrates a second example in which a query 20552 forcreating a new data table is provided to an aggregator 20140, whichcauses an edge device 20120A to store the data table in edge storage20122A. The example query 20502A of FIG. 170A uses a non-unique shardalgorithm (labelled “local_genetic” in the example query) and with anedge device identifier (labelled “edge_device_id” in the example query)as an argument to the shard algorithm. In the illustrated example, anytwo events by the same edge device 20120 may be on the same partition.In embodiments, this property may be advantageously used for efficientquery execution (e.g., of an example query including a COUNT(DISTINCTedge_device_id) portion) because any two equal (non-distinct)edge_device_id values may be guaranteed to be on the same partition(e.g., in the same edge storage). In these embodiments, data for edgedevices can be stored in duplicate locations and the dynamic ledger maybe applied to distribute processing while ensuring non-duplicatedresults.

FIGS. 171A-B illustrate example data flows 20600, 20650 for queryingdata stored in the distributed database using a query language (e.g.,EDQL) as described herein. In the illustrated examples, queries 20502are formatted in an SQL format with custom DML extensions (e.g., EDQLformat). However, it should be understood that, in some embodiments, thequeries may be formatted in other query languages. The example dataflows 20600, 20650 illustrate specific DML concepts that may be used toquery a distributed database in the context of the edge distributedsystem as described herein.

In embodiments, the portioning of a distributed table may affect theperformance of some kinds of queries (e.g., EDQL queries with a SELECTstatement). In embodiments, an EXPLAIN statement/command may be used toexamine query plans corresponding to a query as generated by a queryplanning module 20356, a query execution module 20308, and/or an edgestorage module 20310.

In embodiments, a query language with DML extensions for a distributeddatabase (e.g., EDQL) may leverage one or more of a variety of shardalgorithms. In embodiments, the selection of shard algorithm maydetermine overall location of data and thus query performance. Inembodiments, shard algorithms may include one or more of a distributedlogical algorithm (e.g., an algorithm for the partitioning of datawithin the edge cluster based on a set of logical rules, such as columnvalues), a local neural algorithm (e.g., an algorithm for thepartitioning of data that is local to a specific cluster and is based ona neural network), and/or a local genetic algorithm (e.g., an algorithmfor the partitioning of data that is local to a specific cluster and isbased on a genetic algorithms network). In embodiments, EDQL DMLcommands may be used to communicate directly with edge devices 20120 tofind data location.

FIG. 171A illustrates an example data flow 20600 in which a query may bedirected to a single partition. For example, if an equality is specifiedon every column in the shard algorithm parameters, then a device thatreceives the query (e.g., an aggregator 20140) will direct the query toexactly one partition. In the illustrated embodiment, the queries 20602Aand 20602B have been directed to an edge device 20120, which maycorrespond to a specified “edge_node_id” value of “42” as shown in theillustrated example queries 20602A, 20602B. In other words, the queriesare directed to the edge device 20120 that matches the node identifierspecified in the query (in the example case, a single edge device20120).

In embodiments, many queries may not correspond to the pattern of FIG.171A, where the query corresponds to a single partition. In these cases,for example, an aggregator 20140 may send the queries to severalpartitions (e.g., every partition in a particular local cluster) forintermediate results, then stitch them together.

FIG. 171B illustrates an example data flow 20650 in which examplequeries 20652A, 20652B match more than a single partition. In theillustrated example, the queries 20652A, 20652B are not specified to asingle partition (e.g., edge device or edge storage), but the queriesmay be received by a single edge device 20120. In such a case, severalbehaviors are possible. In some embodiments, the edge device 20120(and/or a connected aggregator 20140) may execute either or both of themethods 20400, 20430 of FIGS. 169A, 169B in order to cause the query tobe stored on the dynamic ledger 20150, thus allowing the queries 20652A,20652B to be distributed to multiple edge devices 20120 (e.g., as apending data request 20268). Additionally or alternatively, the queries20652A, 20652B may be directly sent (e.g., by an aggregator 20140) tomultiple partitions in the distributed database (and/or local clusterthereof). In these embodiments, each edge node 20120 may use its part ofa secondary index to speed up the (e.g., query 20652A, which matches asecondary index). Thus, although the overall performance of the querymay be dictated by the seek and scan time of these indexes, sending thequery widely in the cluster can increase the variance (and thereforeoverall latency) of the query. Furthermore, an optimization (e.g., asimplemented by an aggregator 20140) may be to prioritize thedistribution of the data to other partitions to facilitate localprocessing, for example if two edge nodes require shared data and thelatency between them is low (e.g., such that local replication of datamay be cost effective).

Additionally or alternatively, queries that do not match any index(e.g., query 20652B) may cause wide distribution of the query andpotential auto rearrangement to local edge devices 20120. From theperspective of the edge devices 20120 and/or aggregators 20140, thesequeries are similar to queries that match a secondary index (e.g., query20652A), although they may have a larger local calculation cost (e.g., alocal table scan).

In embodiments, aggregators 20140 and/or other devices within thedistributed database may perform aggregation in various manners. Forexample, calculations on a numerator sum (e.g., SUM, COUNT, AVGstatements and the like) may be converted (e.g., toSUM(expr)/COUNT(expr) and the like). Additionally or alternatively,calculations that require a complete knowledge of the dataset (e.g.,COUNT DISTINCT, MEDIAN statements) may be handled in various ways. Inthe case of non-set-based calculations (e.g., calculations with SUM orCOUNT statements), the aggregations can be distributed in the clusterand aggregated dynamically efficiently. Additionally or alternatively,in the case of complete knowledge aggregations, approximation algorithmsmay be used based on the underlying data structure (e.g., accepting theresults can be within tolerances of error) to provide for distributionof a query. For example, a sample of data may be used to determine themedian value and, provided that distributed datasets have a similarshape, query results may be efficiently calculated. Additionally oralternatively, COUNT DISTINCT results may be estimated by probabilitymodels showing the expected number of distinct values based on theoverall size of the dataset.

In embodiments, ORDER BY queries may leverage ordering at the edge nodeto provide for final merge only requirements on data processing at theaggregate level (e.g., at an aggregator 20140).

In embodiments, for aggregate queries, query results may be based on afinal output of one or more edge nodes and/or may be based on estimatesfrom information stored on the dynamic ledger 20150, such as probabilitydistribution models 20260. Additionally or alternatively, associativeresults may be calculated at the final node (e.g., the edge device 20120and/or aggregator 20140 that received the query) to allow for dynamicdata aggregation.

In embodiments, GROUP BY queries may rely on distributions of differentkinds of aggregates and/or edge device aggregation to allow for thefinal results to be calculated.

In embodiments, HAVING queries may be based on partition algorithm logicor summary frequency analysis and/or may be calculated in edge deviceswhere there is partition separation and/or at a final node.

FIGS. 172A-C illustrate example data flows 20700, 20740, 20780 forcreating and query tables of the distributed database using a querylanguage (e.g., EDQL) as described herein. In the illustrated examples,queries are formatted in an SQL format with custom and DDL and DMLextensions (e.g., EDQL format). However, it should be understood that,in some embodiments, the queries may be formatted in other querylanguages. The example data flows 20700, 20740, 20780 illustratespecific concepts related to distributed joins and reference tables thatmay be used to query a distributed database in the context of the edgedistributed system as described herein.

FIG. 172A illustrates several example queries 20702A, 20702B, 20702Cthat create distributed tables linked by reference joins. For example, afirst query 20702A may create a first table (e.g., a table labelled“edge_devices”), a second query 20702B may create a second table (e.g.,a table labelled “events”) that may use a primary key of the first tableas a parameter for a shard algorithm (e.g., edge_device_id), and a thirdquery may create a third table (e.g., a table labelled “users”) that mayuse a different shard algorithm parameter (e.g., account id). Inembodiments, tables constructed in this manner may be reference tables20216 that may be distributed to edge devices 20120 and/or referencetables 20264 that may be stored on a dynamic ledger 20150 (e.g., by anaggregator 20140).

After creation of the distributed tables and/or reference tables asshown in FIG. 172A, distributed join queries may be executed asillustrated by the example data flows 20740, 20780 shown in FIGS. 172B,172C. For example, the query 20742 may be executed against an edgedevice 20120 (e.g., edge device 20120A) as shown at FIG. 172B. In thisexample, the query 20742 specifies a SELECT on a first table (e.g.,“edge_devices” with an INNER JOIN involving two tables (e.g., the“edge_devices” table and an “events” table) on a parameter (e.g., the“id” parameter of both tables). In the illustrated embodiment, the edgedevice 20120A may be able to at least partially respond to the querywithout any network overhead because the join may be executed locally.Distribution of the query 20742 to other edge devices (e.g., to device20120B) may be handled as described elsewhere herein. For example, theedge device 20120A may execute method 20400 to cause the query 20742 tobe posted on a dynamic ledger 20150, which may cause the edge device20120B to execute the query 20742. Additionally or alternatively, anaggregator 20140 may handle distribution of the query 20742 to the edgedevices as necessary.

Similarly, FIG. 172C illustrates an example method 20780 for handling aquery 20782. The example query 20782 specifies a SELECT on a first table(e.g., a “users” table) that is joined using an INNER JOIN with a secondtable (e.g., an “events” table) using a shared parameter (e.g., an“account id” parameter of both tables). The query 20782 may be handledefficiently with distribution of the users table (e.g., at least to theedge device 20120A). As in the previous example, distribution of thequery 20782 to other edge devices (e.g., to device 20120B) may behandled as described elsewhere herein. For example, the edge device20120A may execute method 20400 to cause the query 20782 to be posted ona dynamic ledger 20150, which may cause the edge device 20120B toexecute the query 20782. Additionally or alternatively, an aggregator20140 may handle distribution of the query 20782 to the edge devices asnecessary.

In embodiments, distributed database systems as described herein may beused to store and/or retrieve sensor data generated by components of thevalue chain network, such as robotic components or other components thatinclude and/or are associated with sensors. For example, in theseembodiments, the robots themselves and/or various devices associatedwith the robots may act as edge devices and/or aggregators in order tomake various sensor data available to query devices (e.g., variouscontrol system) that wish to obtain and/or analyze the sensor data.

In embodiments, the distributed database systems as described herein maybe used to store and/or retrieve IoT data that may be generated byvarious IoT devices, including smart products and/or other smartdevices. For example, the data stored in the distributed database mayinclude statuses of each smart devices, location of each smart devices,and/or any other data associated with each smart device. Additionally,the IoT/smart devices and/or associated devices may act as the edgedevices and/or aggregators to store and retrieve the data as describedherein.

CONCLUSION

The background description is presented simply for context, and is notnecessarily well-understood, routine, or conventional. Further, thebackground description is not an admission of what does or does notqualify as prior art. In fact, some or all of the background descriptionmay be work attributable to the named inventors that is otherwiseunknown in the art.

Physical (such as spatial and/or electrical) and functionalrelationships between elements (for example, between modules, circuitelements, semiconductor layers, etc.) are described using various terms.Unless explicitly described as being “direct,” when a relationshipbetween first and second elements is described, that relationshipencompasses both (i) a direct relationship where no other interveningelements are present between the first and second elements and (ii) anindirect relationship where one or more intervening elements are presentbetween the first and second elements. Example relationship termsinclude “adjoining,” “transmitting,” “receiving,” “connected,”“engaged,” “coupled,” “adjacent,” “next to,” “on top of,” “above,”“below,” “abutting,” and “disposed.”

The detailed description includes specific examples for illustrationonly, and not to limit the disclosure or its applicability. The examplesare not intended to be an exhaustive list, but instead simplydemonstrate possession by the inventors of the full scope of thecurrently presented and envisioned future claims. Variations,combinations, and equivalents of the examples are within the scope ofthe disclosure. No language in the specification should be construed asindicating that any non-claimed element is essential or critical to thepractice of the disclosure.

The term “exemplary” simply means “example” and does not indicate a bestor preferred example. The term “set” does not necessarily exclude theempty set—in other words, in some circumstances a “set” may have zeroelements. The term “non-empty set” may be used to indicate exclusion ofthe empty set—that is, a non-empty set must have one or more elements.The term “subset” does not necessarily require a proper subset. In otherwords, a “subset” of a first set may be coextensive with (equal to) thefirst set. Further, the term “subset” does not necessarily exclude theempty set—in some circumstances a “subset” may have zero elements.

The phrase “at least one of A, B, and C” should be construed to mean alogical (A OR B OR C), using a non-exclusive logical OR, and should notbe construed to mean “at least one of A, at least one of B, and at leastone of C.” The use of the terms “a,” “an,” “the,” and similar referentsin the context of describing the disclosure and claims encompasses boththe singular and the plural, unless contradicted explicitly or bycontext. Unless otherwise specified, the terms “comprising,” “having,”“with,” “including,” and “containing,” and their variants, areopen-ended terms, meaning “including, but not limited to.”

Each publication referenced in this disclosure, including foreign anddomestic patent applications and patents, is hereby incorporated byreference in its entirety.

Although each of the embodiments is described above as having certainfeatures, any one or more of those features described with respect toany embodiment of the disclosure can be implemented in and/or combinedwith features of any of the other embodiments, even if that combinationis not explicitly described. In other words, the described embodimentsare not mutually exclusive, and permutations of multiple embodimentsremain within the scope of this disclosure.

One or more elements (for example, steps within a method, instructions,actions, or operations) may be executed in a different order (and/orconcurrently) without altering the principles of the present disclosure.Unless technically infeasible, elements described as being in series maybe implemented partially or fully in parallel. Similarly, unlesstechnically infeasible, elements described as being in parallel may beimplemented partially or fully in series.

While the disclosure describes structures corresponding to claimedelements, those elements do not necessarily invoke a means plus functioninterpretation unless they explicitly use the signifier “means for.”Unless otherwise indicated, recitations of ranges of values are merelyintended to serve as a shorthand way of referring individually to eachseparate value falling within the range, and each separate value ishereby incorporated into the specification as if it were individuallyrecited.

While the drawings divide elements of the disclosure into differentfunctional blocks or action blocks, these divisions are for illustrationonly. According to the principles of the present disclosure,functionality can be combined in other ways such that some or allfunctionality from multiple separately-depicted blocks can beimplemented in a single functional block; similarly, functionalitydepicted in a single block may be separated into multiple blocks. Unlessexplicitly stated as mutually exclusive, features depicted in differentdrawings can be combined consistent with the principles of the presentdisclosure.

In the drawings, reference numbers may be reused to identify identicalelements or may simply identify elements that implement similarfunctionality. Numbering or other labeling of instructions or methodsteps is done for convenient reference, not to indicate a fixed order.In the drawings, the direction of an arrow, as indicated by thearrowhead, generally demonstrates the flow of information (such as dataor instructions) that is of interest to the illustration. For example,when element A and element B exchange a variety of information butinformation transmitted from element A to element B is relevant to theillustration, the arrow may point from element A to element B. Thisunidirectional arrow does not imply that no other information istransmitted from element B to element A. As just one example, forinformation sent from element A to element B, element B may sendrequests and/or acknowledgements to element A.

A special-purpose system includes hardware and/or software and may bedescribed in terms of an apparatus, a method, or a computer-readablemedium. In various embodiments, functionality may be apportioneddifferently between software and hardware. For example, somefunctionality may be implemented by hardware in one embodiment and bysoftware in another embodiment. Further, software may be encoded byhardware structures, and hardware may be defined by software, such as insoftware-defined networking or software-defined radio.

In this application, including the claims, the term module refers to aspecial-purpose system. The module may be implemented by one or morespecial-purpose systems. The one or more special-purpose systems mayalso implement some or all of the other modules. In this application,including the claims, the term module may be replaced with the termscontroller or circuit. In this application, including the claims, theterm platform refers to one or more modules that offer a set offunctions. In this application, including the claims, the term systemmay be used interchangeably with module or with the term special-purposesystem.

The special-purpose system may be directed or controlled by an operator.The special-purpose system may be hosted by one or more of assets ownedby the operator, assets leased by the operator, and third-party assets.The assets may be referred to as a private, community, or hybrid cloudcomputing network or cloud computing environment. For example, thespecial-purpose system may be partially or fully hosted by a third partyoffering software as a service (SaaS), platform as a service (PaaS),and/or infrastructure as a service (IaaS). The special-purpose systemmay be implemented using agile development and operations (DevOps)principles. In embodiments, some or all of the special-purpose systemmay be implemented in a multiple-environment architecture. For example,the multiple environments may include one or more productionenvironments, one or more integration environments, one or moredevelopment environments, etc.

A special-purpose system may be partially or fully implemented using orby a mobile device. Examples of mobile devices include navigationdevices, cell phones, smart phones, mobile phones, mobile personaldigital assistants, palmtops, netbooks, pagers, electronic book readers,tablets, music players, etc. A special-purpose system may be partiallyor fully implemented using or by a network device. Examples of networkdevices include switches, routers, firewalls, gateways, hubs, basestations, access points, repeaters, head-ends, user equipment, cellsites, antennas, towers, etc.

A special-purpose system may be partially or fully implemented using acomputer having a variety of form factors and other characteristics. Forexample, the computer may be characterized as a personal computer, as aserver, etc. The computer may be portable, as in the case of a laptop,netbook, etc. The computer may or may not have any output device, suchas a monitor, line printer, liquid crystal display (LCD), light emittingdiodes (LEDs), etc. The computer may or may not have any input device,such as a keyboard, mouse, touchpad, trackpad, computer vision system,barcode scanner, button array, etc. The computer may run ageneral-purpose operating system, such as the WINDOWS operating systemfrom Microsoft Corporation, the MACOS operating system from Apple, Inc.,or a variant of the LINUX operating system. Examples of servers includea file server, print server, domain server, internet server, intranetserver, cloud server, infrastructure-as-a-service server,platform-as-a-service server, web server, secondary server, host server,distributed server, failover server, and backup server.

The term hardware encompasses components such as processing hardware,storage hardware, networking hardware, and other general-purpose andspecial-purpose components. Note that these are not mutually-exclusivecategories. For example, processing hardware may integrate storagehardware and vice versa.

Examples of a component are integrated circuits (ICs), applicationspecific integrated circuit (ASICs), digital circuit elements, analogcircuit elements, combinational logic circuits, gate arrays such asfield programmable gate arrays (FPGAs), digital signal processors(DSPs), complex programmable logic devices (CPLDs), etc.

Multiple components of the hardware may be integrated, such as on asingle die, in a single package, or on a single printed circuit board orlogic board. For example, multiple components of the hardware may beimplemented as a system-on-chip. A component, or a set of integratedcomponents, may be referred to as a chip, chipset, chiplet, or chipstack. Examples of a system-on-chip include a radio frequency (RF)system-on-chip, an artificial intelligence (AI) system-on-chip, a videoprocessing system-on-chip, an organ-on-chip, a quantum algorithmsystem-on-chip, etc.

The hardware may integrate and/or receive signals from sensors. Thesensors may allow observation and measurement of conditions includingtemperature, pressure, wear, light, humidity, deformation, expansion,contraction, deflection, bending, stress, strain, load-bearing,shrinkage, power, energy, mass, location, temperature, humidity,pressure, viscosity, liquid flow, chemical/gas presence, sound, and airquality. A sensor may include image and/or video capture in visibleand/or non-visible (such as thermal) wavelengths, such as acharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) sensor.

Examples of processing hardware include a central processing unit (CPU),a graphics processing unit (GPU), an approximate computing processor, aquantum computing processor, a parallel computing processor, a neuralnetwork processor, a signal processor, a digital processor, a dataprocessor, an embedded processor, a microprocessor, and a co-processor.The co-processor may provide additional processing functions and/oroptimizations, such as for speed or power consumption. Examples of aco-processor include a math co-processor, a graphics co-processor, acommunication co-processor, a video co-processor, and an artificialintelligence (AI) co-processor.

The processor may enable execution of multiple threads. These multiplethreads may correspond to different programs. In various embodiments, asingle program may be implemented as multiple threads by the programmeror may be decomposed into multiple threads by the processing hardware.The threads may be executed simultaneously to enhance the performance ofthe processor and to facilitate simultaneous operations of theapplication. A processor may be implemented as a packaged semiconductordie. The die includes one or more processing cores and may includeadditional functional blocks, such as cache. In various embodiments, theprocessor may be implemented by multiple dies, which may be combined ina single package or packaged separately.

The networking hardware may include one or more interface circuits. Insome examples, the interface circuit(s) may implement wired or wirelessinterfaces that connect, directly or indirectly, to one or morenetworks. Examples of networks include a cellular network, a local areanetwork (LAN), a wireless personal area network (WPAN), a metropolitanarea network (MAN), and/or a wide area network (WAN). The networks mayinclude one or more of point-to-point and mesh technologies. Datatransmitted or received by the networking components may traverse thesame or different networks. Networks may be connected to each other overa WAN or point-to-point leased lines using technologies such asMultiprotocol Label Switching (MPLS) and virtual private networks(VPNs).

Examples of cellular networks include GSM, GPRS, 3G, 4G, 5G, LTE, andEVDO. The cellular network may be implemented using frequency divisionmultiple access (FDMA) network or code division multiple access (CDMA)network. Examples of a LAN are Institute of Electrical and ElectronicsEngineers (IEEE) Standard 802.11-2020 (also known as the WIFI wirelessnetworking standard) and IEEE Standard 802.3-2018 (also known as theETHERNET wired networking standard). Examples of a WPAN include IEEEStandard 802.15.4, including the ZIGBEE standard from the ZigBeeAlliance. Further examples of a WPAN include the BLUETOOTH wirelessnetworking standard, including Core Specification versions 3.0, 4.0,4.1, 4.2, 5.0, and 5.1 from the Bluetooth Special Interest Group (SIG).A WAN may also be referred to as a distributed communications system(DCS). One example of a WAN is the internet.

Storage hardware is or includes a computer-readable medium. The termcomputer-readable medium, as used in this disclosure, encompasses bothnonvolatile storage and volatile storage, such as dynamic random accessmemory (DRAM). The term computer-readable medium only excludestransitory electrical or electromagnetic signals propagating through amedium (such as on a carrier wave). A computer-readable medium in thisdisclosure is therefore non-transitory, and may also be considered to betangible.

Examples of storage implemented by the storage hardware include adatabase (such as a relational database or a NoSQL database), a datastore, a data lake, a column store, a data warehouse. Example of storagehardware include nonvolatile memory devices, volatile memory devices,magnetic storage media, a storage area network (SAN), network-attachedstorage (NAS), optical storage media, printed media (such as bar codesand magnetic ink), and paper media (such as punch cards and paper tape).The storage hardware may include cache memory, which may be collocatedwith or integrated with processing hardware. Storage hardware may haveread-only, write-once, or read/write properties. Storage hardware may berandom access or sequential access. Storage hardware may belocation-addressable, file-addressable, and/or content-addressable.

Example of nonvolatile memory devices include flash memory (includingNAND and NOR technologies), solid state drives (SSDs), an erasableprogrammable read-only memory device such as an electrically erasableprogrammable read-only memory (EEPROM) device, and a mask read-onlymemory device (ROM). Example of volatile memory devices includeprocessor registers and random access memory (RAM), such as static RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), synchronousgraphics RAM (SGRAM), and video RAM (VRAM). Example of magnetic storagemedia include analog magnetic tape, digital magnetic tape, and rotatinghard disk drive (HDDs). Examples of optical storage media include a CD(such as a CD-R, CD-RW, or CD-ROM), a DVD, a Blu-ray disc, and an UltraHD Blu-ray disc.

Examples of storage implemented by the storage hardware include adistributed ledger, such as a permissioned or permissionless blockchain.Entities recording transactions, such as in a blockchain, may reachconsensus using an algorithm such as proof-of-stake, proof-of-work, andproof-of-storage. Elements of the present disclosure may be representedby or encoded as non-fungible tokens (NFTs). Ownership rights related tothe non-fungible tokens may be recorded in or referenced by adistributed ledger. Transactions initiated by or relevant to the presentdisclosure may use one or both of fiat currency and cryptocurrencies,examples of which include bitcoin and ether. Some or all features ofhardware may be defined using a language for hardware description, suchas IEEE Standard 1364-2005 (commonly called “Verilog”) and IEEE Standard1076-2008 (commonly called “VHDL”). The hardware description languagemay be used to manufacture and/or program hardware.

A special-purpose system may be distributed across multiple differentsoftware and hardware entities. Communication within a special-purposesystem and between special-purpose systems may be performed usingnetworking hardware. The distribution may vary across embodiments andmay vary over time. For example, the distribution may vary based ondemand, with additional hardware and/or software entities invoked tohandle higher demand. In various embodiments, a load balancer may directrequests to one of multiple instantiations of the special purposesystem. The hardware and/or software entities may be physically distinctand/or may share some hardware and/or software, such as in a virtualizedenvironment. Multiple hardware entities may be referred to as a serverrack, server farm, data center, etc.

Software includes instructions that are machine-readable and/orexecutable. Instructions may be logically grouped into programs, codes,methods, steps, actions, routines, functions, libraries, objects,classes, etc. Software may be stored by storage hardware or encoded inother hardware. Software encompasses (i) descriptive text to be parsed,such as HTML (hypertext markup language), XML (extensible markuplanguage), and JSON (JavaScript Object Notation), (ii) assembly code,(iii) object code generated from source code by a compiler, (iv) sourcecode for execution by an interpreter, (v) bytecode, (vi) source code forcompilation and execution by a just-in-time compiler, etc. As examplesonly, source code may be written using syntax from languages includingC, C++, JavaScript, Java, Python, R, etc.

Software also includes data. However, data and instructions are notmutually-exclusive categories. In various embodiments, the instructionsmay be used as data in one or more operations. As another example,instructions may be derived from data. The functional blocks andflowchart elements in this disclosure serve as software specifications,which can be translated into software by the routine work of a skilledtechnician or programmer. Software may include and/or rely on firmware,processor microcode, an operating system (OS), a basic input/outputsystem (BIOS), application programming interfaces (APIs), libraries suchas dynamic-link libraries (DLLs), device drivers, hypervisors, userapplications, background services, background applications, etc.Software includes native applications and web applications. For example,a web application may be served to a device through a browser usinghypertext markup language 5th revision (HTML5).

Software may include artificial intelligence systems, which may includemachine learning or other computational intelligence. For example,artificial intelligence may include one or more models used for one ormore problem domains. When presented with many data features,identification of a subset of features that are relevant to a problemdomain may improve prediction accuracy, reduce storage space, andincrease processing speed. This identification may be referred to asfeature engineering. Feature engineering may be performed by users ormay only be guided by users. In various implementations, a machinelearning system may computationally identify relevant features, such asby performing singular value decomposition on the contributions ofdifferent features to outputs.

Examples of the models include recurrent neural networks (RNNs) such aslong short-term memory (LSTM), deep learning models such astransformers, decision trees, support-vector machines, geneticalgorithms, Bayesian networks, and regression analysis. Examples ofsystems based on a transformer model include bidirectional encoderrepresentations from transformers (BERT) and generative pre-trainedtransformer (GPT). Training a machine-learning model may includesupervised learning (for example, based on labelled input data),unsupervised learning, and reinforcement learning. In variousembodiments, a machine-learning model may be pre-trained by theiroperator or by a third party. Problem domains include nearly anysituation where structured data can be collected, and includes naturallanguage processing (NLP), computer vision (CV), classification, imagerecognition, etc.

Some or all of the software may run in a virtual environment rather thandirectly on hardware. The virtual environment may include a hypervisor,emulator, sandbox, container engine, etc. The software may be built as avirtual machine, a container, etc. Virtualized resources may becontrolled using, for example, a DOCKER container platform, a pivotalcloud foundry (PCF) platform, etc.

In a client-server model, some of the software executes on firsthardware identified functionally as a server, while other of thesoftware executes on second hardware identified functionally as aclient. The identity of the client and server is not fixed: for somefunctionality, the first hardware may act as the server while for otherfunctionality, the first hardware may act as the client. In differentembodiments and in different scenarios, functionality may be shiftedbetween the client and the server. In one dynamic example, somefunctionality normally performed by the second hardware is shifted tothe first hardware when the second hardware has less capability. Invarious embodiments, the term “local” may be used in place of “client,”and the term “remote” may be used in place of “server.”

Some or all of the software may be logically partitioned intomicroservices. Each microservice offers a reduced subset offunctionality. In various embodiments, each microservice may be scaledindependently depending on load, either by devoting more resources tothe microservice or by instantiating more instances of the microservice.In various embodiments, functionality offered by one or moremicroservices may be combined with each other and/or with other softwarenot adhering to a microservices model.

Some or all of the software may be arranged logically into layers. In alayered architecture, a second layer may be logically placed between afirst layer and a third layer. The first layer and the third layer wouldthen generally interact with the second layer and not with each other.In various embodiments, this is not strictly enforced—that is, somedirect communication may occur between the first and third layers.

1. A robot fleet management platform comprising: a computer-readablestorage system that stores a resources data store that maintains: arobot inventory that indicates a plurality of robots that can beassigned to a robot fleet, and for each respective robot, the robotinventory indicates a set of baseline features of each robot and arespective status of each robot, wherein the robot inventory of robotsincludes a plurality of multi-purpose robots that can be configured fordifferent tasks and different environments; and a components inventorythat indicates different components that can be provisioned to one ormore multi-purpose robots, and for each component, a respective set ofextended capabilities corresponding to each component and a respectivestatus of each component; and a set of one or more processors thatexecute a set of computer-readable instructions, wherein the set of oneor more processors collectively: receive a request for a robotic fleetto perform a job; determine a job definition data structure based on therequest, wherein the job definition data structure defines a set oftasks that are to be performed in performance of the job; determine arobot fleet configuration data structure corresponding to the job basedon the set of tasks and the robot inventory, wherein the robot fleetconfiguration data structure assigns a plurality of robots selected fromthe robot inventory to the set of tasks defined in the job definitiondata structure, and the plurality of robots includes one or moreassigned multi-purpose robots; determine a respective configuration foreach respective one or more assigned multi-purpose robots based on therespective set of tasks that is assigned to the one or more assignedmulti-purpose robots and the components inventory; configure the one ormore assigned multi-purpose robots based on the respectiveconfiguration; and deploy the robotic fleet including the one or moreassigned multi-purpose robots to perform the job.
 2. The robot fleetmanagement platform of claim 1 wherein: the robot inventory includesspecial purpose robots, and the determination of the robot fleetconfiguration data structure is further based on an available inventoryof the special purpose robots.
 3. The robot fleet management platform ofclaim 1 wherein the determination of the robot fleet configuration datastructure is further based on at least one of: an environment of thejob, a budget for the job, or a timeline for completing the job.
 4. Therobot fleet management platform of claim 1 wherein the determination ofthe respective configuration for each respective assigned multi-purposerobot is further based on at least one of: an environment of the job, abudget for the job, or a timeline for completing the job.
 5. The robotfleet management platform of claim 1 wherein the configuration of theone or more assigned multi-purpose robots includes accessing a robotmodule system via at least one of a physical interface module and acontrol interface module.
 6. The robot fleet management platform ofclaim 1 wherein: the configuration of the one or more assignedmulti-purpose robots includes a configuration of one or more modules ofa robot baseline system; and the one or more modules include at leastone of: an energy storage and power distribution system, anelectromechanical and electro-fluidic system, a transport system, or avision and sensing system.
 7. The robot fleet management platform ofclaim 1 wherein the configuration of the one or more assignedmulti-purpose robots includes a configuration of a three-dimensional(3D) printing system to produce at least one robot hardware module. 8.The robot fleet management platform of claim 1 wherein the configurationof the one or more assigned multi-purpose robots is based on one or morecharacteristics of a target operating environment, and wherein thetarget operating environment is at least one of: land-based, sea-based,submerged, in-flight, subterranean, or below-freezing ambienttemperature.
 9. The robot fleet management platform of claim 1 whereinthe configuration of the one or more assigned multi-purpose robotsincludes a configuration of an energy storage and power distributionsystem to utilize two or more distinct power sources based on an aspectof at least one of a task or an operating environment.
 10. The robotfleet management platform of claim 9 wherein: a first distinct powersource of the two or more distinct power sources is a mobile powersource of the one or more assigned multi-purpose robots; and a seconddistinct power source of the two or more distinct power sources is afixed position power source that provides power to the robot via awireless power signal.
 11. A method for configuring a multi-purposerobot of a fleet of robots, the method comprising: receiving a requestfor a robotic fleet to perform a job; defining a set of tasks that areto be performed in performance of the job; assigning a plurality ofrobots selected from a robot inventory to the set of tasks based on theset of tasks and a robot inventory data structure that indicates aplurality of robots that can be assigned to a robot fleet, wherein: foreach respective robot, the robot inventory indicates a set of baselinefeatures of the robot and a respective status of the robot, and theplurality of robots includes one or more assigned multi-purpose robotsthat can be configured for different tasks and different environments;determining a respective configuration for each respective assignedmulti-purpose robot based on the respective set of tasks that isassigned to the one or more assigned multi-purpose robots; indicatingdifferent components that can be provisioned to one or moremulti-purpose robots, and for each component, a respective set ofextended capabilities corresponding to each component and a respectivestatus of each component; configuring the one or more assignedmulti-purpose robots based on the respective configuration; anddeploying the robotic fleet including the one or more assignedmulti-purpose robots to perform the job.
 12. The method of claim 11wherein the assigning the plurality of robots selected from the robotinventory is further based on at least one of: an environment of thejob, a budget for the job, or a timeline for completing the job.
 13. Themethod of claim 11 wherein the configuring the one or more assignedmulti-purpose robots further comprises configuring at least one robotsystem including at least one of a robot baseline system, a modulesystem, a robot control system, or a robot security system.
 14. Themethod of claim 11 wherein the configuring the one or more assignedmulti-purpose robots further comprises configuring at least one of asoftware robot module, a non-interchangeable hardware robot module, oran interchangeable hardware robot module.
 15. The method of claim 11wherein the configuring the one or more assigned multi-purpose robotsfurther comprises accessing a robot module system via at least one of aphysical interface module and a control interface module.
 16. The methodof claim 11 wherein the configuring the one or more assignedmulti-purpose robots further comprises configuring a propulsion systemof each robot to adaptably utilize one or more legs for locomotion. 17.The method of claim 11 wherein: the configuring the one or more assignedmulti-purpose robots further comprises provisioning one or more modulesidentified in a job execution plan to the one or more assignedmulti-purpose robots, and the one or more modules is at least one of ahardware module or software module.
 18. The method of claim 11 whereinthe configuring the one or more assigned multi-purpose robots furthercomprises provisioning at least one of: appendages, sensor sets,chipsets, or motive adaptors to the multi-purpose robot based on atleast one task in a set of target tasks for the robot that areidentified in a job execution plan.
 19. The method of claim 11 whereinthe configuring the one or more assigned multi-purpose robots furthercomprises analyzing a job execution plan that defines a fleet of robotsand configuring at least one multi-purpose robot of the fleet of robots.20. The method of claim 11 wherein the configuring the one or moreassigned multi-purpose robots further comprises provisioning a localmanager capability that enables the multi-purpose robot to control oneor more robots.